2023-07-24 12:14:47,955 INFO [main] hbase.HBaseClassTestRule(94): Test class org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure timeout: 13 mins 2023-07-24 12:14:48,272 DEBUG [main] hbase.HBaseTestingUtility(342): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234 2023-07-24 12:14:48,286 INFO [Time-limited test] hbase.HBaseTestingUtility(1068): Starting up minicluster with option: StartMiniClusterOption{numMasters=1, masterClass=null, numRegionServers=3, rsPorts=, rsClass=null, numDataNodes=3, dataNodeHosts=null, numZkServers=1, createRootDir=false, createWALDir=false} 2023-07-24 12:14:48,287 INFO [Time-limited test] hbase.HBaseZKTestingUtility(82): Created new mini-cluster data directory: /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d, deleteOnExit=true 2023-07-24 12:14:48,287 INFO [Time-limited test] hbase.HBaseTestingUtility(1082): STARTING DFS 2023-07-24 12:14:48,288 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting test.cache.data to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/test.cache.data in system properties and HBase conf 2023-07-24 12:14:48,288 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.tmp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/hadoop.tmp.dir in system properties and HBase conf 2023-07-24 12:14:48,289 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting hadoop.log.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/hadoop.log.dir in system properties and HBase conf 2023-07-24 12:14:48,289 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.local.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/mapreduce.cluster.local.dir in system properties and HBase conf 2023-07-24 12:14:48,290 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting mapreduce.cluster.temp.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/mapreduce.cluster.temp.dir in system properties and HBase conf 2023-07-24 12:14:48,290 INFO [Time-limited test] hbase.HBaseTestingUtility(759): read short circuit is OFF 2023-07-24 12:14:48,410 WARN [Time-limited test] util.NativeCodeLoader(62): Unable to load native-hadoop library for your platform... using builtin-java classes where applicable 2023-07-24 12:14:48,867 DEBUG [Time-limited test] fs.HFileSystem(308): The file system is not a DistributedFileSystem. Skipping on block location reordering 2023-07-24 12:14:48,873 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-labels.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/yarn.node-labels.fs-store.root-dir in system properties and HBase conf 2023-07-24 12:14:48,873 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.node-attribute.fs-store.root-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/yarn.node-attribute.fs-store.root-dir in system properties and HBase conf 2023-07-24 12:14:48,874 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.log-dirs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/yarn.nodemanager.log-dirs in system properties and HBase conf 2023-07-24 12:14:48,874 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-24 12:14:48,875 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.active-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/yarn.timeline-service.entity-group-fs-store.active-dir in system properties and HBase conf 2023-07-24 12:14:48,875 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.timeline-service.entity-group-fs-store.done-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/yarn.timeline-service.entity-group-fs-store.done-dir in system properties and HBase conf 2023-07-24 12:14:48,876 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting yarn.nodemanager.remote-app-log-dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/yarn.nodemanager.remote-app-log-dir in system properties and HBase conf 2023-07-24 12:14:48,876 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-24 12:14:48,876 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.datanode.shared.file.descriptor.paths to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/dfs.datanode.shared.file.descriptor.paths in system properties and HBase conf 2023-07-24 12:14:48,877 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting nfs.dump.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/nfs.dump.dir in system properties and HBase conf 2023-07-24 12:14:48,877 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting java.io.tmpdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/java.io.tmpdir in system properties and HBase conf 2023-07-24 12:14:48,878 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.journalnode.edits.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/dfs.journalnode.edits.dir in system properties and HBase conf 2023-07-24 12:14:48,878 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting dfs.provided.aliasmap.inmemory.leveldb.dir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/dfs.provided.aliasmap.inmemory.leveldb.dir in system properties and HBase conf 2023-07-24 12:14:48,879 INFO [Time-limited test] hbase.HBaseTestingUtility(772): Setting fs.s3a.committer.staging.tmp.path to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/fs.s3a.committer.staging.tmp.path in system properties and HBase conf Formatting using clusterid: testClusterID 2023-07-24 12:14:49,534 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-24 12:14:49,538 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-24 12:14:49,871 WARN [Time-limited test] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-namenode.properties,hadoop-metrics2.properties 2023-07-24 12:14:50,040 INFO [Time-limited test] log.Slf4jLog(67): Logging to org.slf4j.impl.Reload4jLoggerAdapter(org.mortbay.log) via org.mortbay.log.Slf4jLog 2023-07-24 12:14:50,058 WARN [Time-limited test] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-24 12:14:50,106 INFO [Time-limited test] log.Slf4jLog(67): jetty-6.1.26 2023-07-24 12:14:50,137 INFO [Time-limited test] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/hdfs to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/java.io.tmpdir/Jetty_localhost_42649_hdfs____eo6nxj/webapp 2023-07-24 12:14:50,299 INFO [Time-limited test] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:42649 2023-07-24 12:14:50,320 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.heartbeat.interval(3) assuming SECONDS 2023-07-24 12:14:50,320 WARN [Time-limited test] conf.Configuration(1701): No unit for dfs.namenode.safemode.extension(0) assuming MILLISECONDS 2023-07-24 12:14:50,912 WARN [Listener at localhost/33821] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-24 12:14:50,990 WARN [Listener at localhost/33821] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-24 12:14:51,013 WARN [Listener at localhost/33821] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-24 12:14:51,019 INFO [Listener at localhost/33821] log.Slf4jLog(67): jetty-6.1.26 2023-07-24 12:14:51,023 INFO [Listener at localhost/33821] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/java.io.tmpdir/Jetty_localhost_40703_datanode____v1ndr9/webapp 2023-07-24 12:14:51,128 INFO [Listener at localhost/33821] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:40703 2023-07-24 12:14:51,483 WARN [Listener at localhost/39863] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-24 12:14:51,504 WARN [Listener at localhost/39863] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-24 12:14:51,511 WARN [Listener at localhost/39863] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-24 12:14:51,513 INFO [Listener at localhost/39863] log.Slf4jLog(67): jetty-6.1.26 2023-07-24 12:14:51,521 INFO [Listener at localhost/39863] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/java.io.tmpdir/Jetty_localhost_45255_datanode____pr1y1k/webapp 2023-07-24 12:14:51,633 INFO [Listener at localhost/39863] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45255 2023-07-24 12:14:51,653 WARN [Listener at localhost/38465] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-24 12:14:51,686 WARN [Listener at localhost/38465] conf.Configuration(1701): No unit for dfs.datanode.outliers.report.interval(1800000) assuming MILLISECONDS 2023-07-24 12:14:51,689 WARN [Listener at localhost/38465] http.HttpRequestLog(97): Jetty request log can only be enabled using Log4j 2023-07-24 12:14:51,691 INFO [Listener at localhost/38465] log.Slf4jLog(67): jetty-6.1.26 2023-07-24 12:14:51,704 INFO [Listener at localhost/38465] log.Slf4jLog(67): Extract jar:file:/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/local-repository/org/apache/hadoop/hadoop-hdfs/2.10.0/hadoop-hdfs-2.10.0-tests.jar!/webapps/datanode to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/java.io.tmpdir/Jetty_localhost_45999_datanode____5d082n/webapp 2023-07-24 12:14:51,830 INFO [Listener at localhost/38465] log.Slf4jLog(67): Started HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:45999 2023-07-24 12:14:51,843 WARN [Listener at localhost/37279] common.MetricsLoggerTask(153): Metrics logging will not be async since the logger is not log4j 2023-07-24 12:14:52,161 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x474812c490474b7d: Processing first storage report for DS-9a5cb7b9-3c18-456e-b03c-b506d2a6d9f9 from datanode 8ac734ec-00fd-4136-ae08-8d364bd1fe4b 2023-07-24 12:14:52,163 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x474812c490474b7d: from storage DS-9a5cb7b9-3c18-456e-b03c-b506d2a6d9f9 node DatanodeRegistration(127.0.0.1:35821, datanodeUuid=8ac734ec-00fd-4136-ae08-8d364bd1fe4b, infoPort=36957, infoSecurePort=0, ipcPort=37279, storageInfo=lv=-57;cid=testClusterID;nsid=1839562947;c=1690200889613), blocks: 0, hasStaleStorage: true, processing time: 2 msecs, invalidatedBlocks: 0 2023-07-24 12:14:52,163 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x41c36fb49b903c40: Processing first storage report for DS-aa809477-162f-4591-b20e-2f3db95c540a from datanode dce5ccf6-5af1-4cab-b72f-523ad9084719 2023-07-24 12:14:52,163 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x41c36fb49b903c40: from storage DS-aa809477-162f-4591-b20e-2f3db95c540a node DatanodeRegistration(127.0.0.1:44861, datanodeUuid=dce5ccf6-5af1-4cab-b72f-523ad9084719, infoPort=36593, infoSecurePort=0, ipcPort=38465, storageInfo=lv=-57;cid=testClusterID;nsid=1839562947;c=1690200889613), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-24 12:14:52,164 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x7a1fd1307668c45e: Processing first storage report for DS-8e7af3e2-677f-457f-972b-9a9656ca819c from datanode 295fb89a-ce04-4534-9158-8d1eb7f1a7e1 2023-07-24 12:14:52,164 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x7a1fd1307668c45e: from storage DS-8e7af3e2-677f-457f-972b-9a9656ca819c node DatanodeRegistration(127.0.0.1:46387, datanodeUuid=295fb89a-ce04-4534-9158-8d1eb7f1a7e1, infoPort=37861, infoSecurePort=0, ipcPort=39863, storageInfo=lv=-57;cid=testClusterID;nsid=1839562947;c=1690200889613), blocks: 0, hasStaleStorage: true, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-24 12:14:52,164 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x474812c490474b7d: Processing first storage report for DS-f0e51610-ff93-4472-965c-008dc7e7a44b from datanode 8ac734ec-00fd-4136-ae08-8d364bd1fe4b 2023-07-24 12:14:52,164 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x474812c490474b7d: from storage DS-f0e51610-ff93-4472-965c-008dc7e7a44b node DatanodeRegistration(127.0.0.1:35821, datanodeUuid=8ac734ec-00fd-4136-ae08-8d364bd1fe4b, infoPort=36957, infoSecurePort=0, ipcPort=37279, storageInfo=lv=-57;cid=testClusterID;nsid=1839562947;c=1690200889613), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-24 12:14:52,165 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x41c36fb49b903c40: Processing first storage report for DS-50e81562-0e80-410a-a309-67ae42beeb98 from datanode dce5ccf6-5af1-4cab-b72f-523ad9084719 2023-07-24 12:14:52,165 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x41c36fb49b903c40: from storage DS-50e81562-0e80-410a-a309-67ae42beeb98 node DatanodeRegistration(127.0.0.1:44861, datanodeUuid=dce5ccf6-5af1-4cab-b72f-523ad9084719, infoPort=36593, infoSecurePort=0, ipcPort=38465, storageInfo=lv=-57;cid=testClusterID;nsid=1839562947;c=1690200889613), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-24 12:14:52,165 INFO [Block report processor] blockmanagement.BlockManager(2202): BLOCK* processReport 0x7a1fd1307668c45e: Processing first storage report for DS-ced6c1dc-7136-480e-a42f-6ca8106a3464 from datanode 295fb89a-ce04-4534-9158-8d1eb7f1a7e1 2023-07-24 12:14:52,165 INFO [Block report processor] blockmanagement.BlockManager(2228): BLOCK* processReport 0x7a1fd1307668c45e: from storage DS-ced6c1dc-7136-480e-a42f-6ca8106a3464 node DatanodeRegistration(127.0.0.1:46387, datanodeUuid=295fb89a-ce04-4534-9158-8d1eb7f1a7e1, infoPort=37861, infoSecurePort=0, ipcPort=39863, storageInfo=lv=-57;cid=testClusterID;nsid=1839562947;c=1690200889613), blocks: 0, hasStaleStorage: false, processing time: 0 msecs, invalidatedBlocks: 0 2023-07-24 12:14:52,345 DEBUG [Listener at localhost/37279] hbase.HBaseTestingUtility(649): Setting hbase.rootdir to /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234 2023-07-24 12:14:52,411 INFO [Listener at localhost/37279] zookeeper.MiniZooKeeperCluster(258): Started connectionTimeout=30000, dir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/zookeeper_0, clientPort=58848, secureClientPort=-1, dataDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/zookeeper_0/version-2, dataDirSize=424 dataLogDir=/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/zookeeper_0/version-2, dataLogSize=424 tickTime=2000, maxClientCnxns=300, minSessionTimeout=4000, maxSessionTimeout=40000, serverId=0 2023-07-24 12:14:52,433 INFO [Listener at localhost/37279] zookeeper.MiniZooKeeperCluster(283): Started MiniZooKeeperCluster and ran 'stat' on client port=58848 2023-07-24 12:14:52,445 INFO [Listener at localhost/37279] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 12:14:52,448 INFO [Listener at localhost/37279] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 12:14:53,146 INFO [Listener at localhost/37279] util.FSUtils(471): Created version file at hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef with version=8 2023-07-24 12:14:53,147 INFO [Listener at localhost/37279] hbase.HBaseTestingUtility(1406): Setting hbase.fs.tmp.dir to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/hbase-staging 2023-07-24 12:14:53,478 INFO [Listener at localhost/37279] metrics.MetricRegistriesLoader(60): Loaded MetricRegistries class org.apache.hadoop.hbase.metrics.impl.MetricRegistriesImpl 2023-07-24 12:14:53,952 INFO [Listener at localhost/37279] client.ConnectionUtils(127): master/jenkins-hbase6:0 server-side Connection retries=45 2023-07-24 12:14:53,984 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 12:14:53,985 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 12:14:53,985 INFO [Listener at localhost/37279] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 12:14:53,985 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 12:14:53,985 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 12:14:54,125 INFO [Listener at localhost/37279] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.MasterService, hbase.pb.RegionServerStatusService, hbase.pb.LockService, hbase.pb.HbckService, hbase.pb.ClientMetaService, hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 12:14:54,203 DEBUG [Listener at localhost/37279] util.ClassSize(228): Using Unsafe to estimate memory layout 2023-07-24 12:14:54,298 INFO [Listener at localhost/37279] ipc.NettyRpcServer(120): Bind to /172.31.2.20:44983 2023-07-24 12:14:54,309 INFO [Listener at localhost/37279] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 12:14:54,310 INFO [Listener at localhost/37279] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 12:14:54,335 INFO [Listener at localhost/37279] zookeeper.RecoverableZooKeeper(93): Process identifier=master:44983 connecting to ZooKeeper ensemble=127.0.0.1:58848 2023-07-24 12:14:54,375 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:449830x0, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 12:14:54,378 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): master:44983-0x100becce0d60000 connected 2023-07-24 12:14:54,404 DEBUG [Listener at localhost/37279] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 12:14:54,405 DEBUG [Listener at localhost/37279] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 12:14:54,408 DEBUG [Listener at localhost/37279] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 12:14:54,416 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=44983 2023-07-24 12:14:54,416 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=44983 2023-07-24 12:14:54,417 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=44983 2023-07-24 12:14:54,417 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=44983 2023-07-24 12:14:54,417 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=44983 2023-07-24 12:14:54,423 INFO [Listener at localhost/37279] master.HMaster(444): hbase.rootdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef, hbase.cluster.distributed=false 2023-07-24 12:14:54,489 INFO [Listener at localhost/37279] client.ConnectionUtils(127): regionserver/jenkins-hbase6:0 server-side Connection retries=45 2023-07-24 12:14:54,489 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 12:14:54,489 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 12:14:54,489 INFO [Listener at localhost/37279] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 12:14:54,489 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 12:14:54,490 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 12:14:54,494 INFO [Listener at localhost/37279] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 12:14:54,497 INFO [Listener at localhost/37279] ipc.NettyRpcServer(120): Bind to /172.31.2.20:41789 2023-07-24 12:14:54,499 INFO [Listener at localhost/37279] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 12:14:54,505 DEBUG [Listener at localhost/37279] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 12:14:54,506 INFO [Listener at localhost/37279] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 12:14:54,508 INFO [Listener at localhost/37279] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 12:14:54,510 INFO [Listener at localhost/37279] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:41789 connecting to ZooKeeper ensemble=127.0.0.1:58848 2023-07-24 12:14:54,514 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:417890x0, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 12:14:54,515 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:41789-0x100becce0d60001 connected 2023-07-24 12:14:54,515 DEBUG [Listener at localhost/37279] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 12:14:54,516 DEBUG [Listener at localhost/37279] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 12:14:54,517 DEBUG [Listener at localhost/37279] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 12:14:54,518 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=41789 2023-07-24 12:14:54,518 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=41789 2023-07-24 12:14:54,519 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=41789 2023-07-24 12:14:54,520 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=41789 2023-07-24 12:14:54,521 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=41789 2023-07-24 12:14:54,533 INFO [Listener at localhost/37279] client.ConnectionUtils(127): regionserver/jenkins-hbase6:0 server-side Connection retries=45 2023-07-24 12:14:54,533 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 12:14:54,533 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 12:14:54,533 INFO [Listener at localhost/37279] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 12:14:54,534 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 12:14:54,534 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 12:14:54,534 INFO [Listener at localhost/37279] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 12:14:54,536 INFO [Listener at localhost/37279] ipc.NettyRpcServer(120): Bind to /172.31.2.20:34575 2023-07-24 12:14:54,536 INFO [Listener at localhost/37279] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 12:14:54,540 DEBUG [Listener at localhost/37279] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 12:14:54,540 INFO [Listener at localhost/37279] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 12:14:54,542 INFO [Listener at localhost/37279] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 12:14:54,544 INFO [Listener at localhost/37279] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:34575 connecting to ZooKeeper ensemble=127.0.0.1:58848 2023-07-24 12:14:54,548 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:345750x0, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 12:14:54,549 DEBUG [Listener at localhost/37279] zookeeper.ZKUtil(164): regionserver:345750x0, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 12:14:54,550 DEBUG [Listener at localhost/37279] zookeeper.ZKUtil(164): regionserver:345750x0, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 12:14:54,551 DEBUG [Listener at localhost/37279] zookeeper.ZKUtil(164): regionserver:345750x0, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 12:14:54,552 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:34575-0x100becce0d60002 connected 2023-07-24 12:14:54,552 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=34575 2023-07-24 12:14:54,552 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=34575 2023-07-24 12:14:54,553 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=34575 2023-07-24 12:14:54,560 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=34575 2023-07-24 12:14:54,561 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=34575 2023-07-24 12:14:54,573 INFO [Listener at localhost/37279] client.ConnectionUtils(127): regionserver/jenkins-hbase6:0 server-side Connection retries=45 2023-07-24 12:14:54,573 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated default.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 12:14:54,573 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated priority.RWQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=2, maxQueueLength=30, handlerCount=3 2023-07-24 12:14:54,573 INFO [Listener at localhost/37279] ipc.RWQueueRpcExecutor(107): priority.RWQ.Fifo writeQueues=1 writeHandlers=1 readQueues=1 readHandlers=2 scanQueues=0 scanHandlers=0 2023-07-24 12:14:54,573 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated replication.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=3 2023-07-24 12:14:54,573 INFO [Listener at localhost/37279] ipc.RpcExecutor(189): Instantiated metaPriority.FPBQ.Fifo with queueClass=class java.util.concurrent.LinkedBlockingQueue; numCallQueues=1, maxQueueLength=30, handlerCount=1 2023-07-24 12:14:54,573 INFO [Listener at localhost/37279] ipc.RpcServerFactory(64): Creating org.apache.hadoop.hbase.ipc.NettyRpcServer hosting hbase.pb.ClientService, hbase.pb.AdminService 2023-07-24 12:14:54,575 INFO [Listener at localhost/37279] ipc.NettyRpcServer(120): Bind to /172.31.2.20:36983 2023-07-24 12:14:54,575 INFO [Listener at localhost/37279] hfile.BlockCacheFactory(142): Allocating BlockCache size=782.40 MB, blockSize=64 KB 2023-07-24 12:14:54,577 DEBUG [Listener at localhost/37279] mob.MobFileCache(120): MobFileCache enabled with cacheSize=1000, evictPeriods=3600sec, evictRemainRatio=0.5 2023-07-24 12:14:54,578 INFO [Listener at localhost/37279] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 12:14:54,579 INFO [Listener at localhost/37279] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 12:14:54,580 INFO [Listener at localhost/37279] zookeeper.RecoverableZooKeeper(93): Process identifier=regionserver:36983 connecting to ZooKeeper ensemble=127.0.0.1:58848 2023-07-24 12:14:54,585 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:369830x0, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=None, state=SyncConnected, path=null 2023-07-24 12:14:54,586 DEBUG [zk-event-processor-pool-0] zookeeper.ZKWatcher(625): regionserver:36983-0x100becce0d60003 connected 2023-07-24 12:14:54,586 DEBUG [Listener at localhost/37279] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 12:14:54,587 DEBUG [Listener at localhost/37279] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 12:14:54,588 DEBUG [Listener at localhost/37279] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/acl 2023-07-24 12:14:54,590 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=default.FPBQ.Fifo, numCallQueues=1, port=36983 2023-07-24 12:14:54,590 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=priority.RWQ.Fifo.write, numCallQueues=1, port=36983 2023-07-24 12:14:54,590 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=priority.RWQ.Fifo.read, numCallQueues=1, port=36983 2023-07-24 12:14:54,591 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=3 with threadPrefix=replication.FPBQ.Fifo, numCallQueues=1, port=36983 2023-07-24 12:14:54,591 DEBUG [Listener at localhost/37279] ipc.RpcExecutor(311): Started handlerCount=1 with threadPrefix=metaPriority.FPBQ.Fifo, numCallQueues=1, port=36983 2023-07-24 12:14:54,593 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.HMaster(2168): Adding backup master ZNode /hbase/backup-masters/jenkins-hbase6.apache.org,44983,1690200893288 2023-07-24 12:14:54,602 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-24 12:14:54,603 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/backup-masters/jenkins-hbase6.apache.org,44983,1690200893288 2023-07-24 12:14:54,621 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 12:14:54,621 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 12:14:54,621 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 12:14:54,621 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/master 2023-07-24 12:14:54,621 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 12:14:54,623 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-24 12:14:54,624 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.ActiveMasterManager(227): Deleting ZNode for /hbase/backup-masters/jenkins-hbase6.apache.org,44983,1690200893288 from backup master directory 2023-07-24 12:14:54,624 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/master 2023-07-24 12:14:54,628 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/backup-masters/jenkins-hbase6.apache.org,44983,1690200893288 2023-07-24 12:14:54,628 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/backup-masters 2023-07-24 12:14:54,629 WARN [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 12:14:54,629 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.ActiveMasterManager(237): Registered as active master=jenkins-hbase6.apache.org,44983,1690200893288 2023-07-24 12:14:54,631 INFO [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating data MemStoreChunkPool with chunk size 2 MB, max count 352, initial count 0 2023-07-24 12:14:54,633 INFO [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.ChunkCreator(497): Allocating index MemStoreChunkPool with chunk size 204.80 KB, max count 391, initial count 0 2023-07-24 12:14:54,730 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] util.FSUtils(620): Created cluster ID file at hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/hbase.id with ID: 8a7413ce-6df9-498a-a28b-69c904dece7d 2023-07-24 12:14:54,769 INFO [master/jenkins-hbase6:0:becomeActiveMaster] fs.HFileSystem(337): Added intercepting call to namenode#getBlockLocations so can do block reordering using class org.apache.hadoop.hbase.fs.HFileSystem$ReorderWALBlocks 2023-07-24 12:14:54,785 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 12:14:54,850 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ReadOnlyZKClient(139): Connect 0x00a8cb5b to 127.0.0.1:58848 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 12:14:54,882 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4696169a, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-24 12:14:54,905 INFO [master/jenkins-hbase6:0:becomeActiveMaster] region.MasterRegion(309): Create or load local region for table 'master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 12:14:54,907 INFO [master/jenkins-hbase6:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(132): Injected flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000 2023-07-24 12:14:54,924 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(264): ClientProtocol::create wrong number of arguments, should be hadoop 3.2 or below 2023-07-24 12:14:54,924 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(270): ClientProtocol::create wrong number of arguments, should be hadoop 2.x 2023-07-24 12:14:54,926 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(279): can not find SHOULD_REPLICATE flag, should be hadoop 2.x java.lang.IllegalArgumentException: No enum constant org.apache.hadoop.fs.CreateFlag.SHOULD_REPLICATE at java.lang.Enum.valueOf(Enum.java:238) at org.apache.hadoop.fs.CreateFlag.valueOf(CreateFlag.java:63) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.loadShouldReplicateFlag(FanOutOneBlockAsyncDFSOutputHelper.java:277) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.(FanOutOneBlockAsyncDFSOutputHelper.java:304) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:139) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-24 12:14:54,932 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(243): No decryptEncryptedDataEncryptionKey method in DFSClient, should be hadoop version with HDFS-12396 java.lang.NoSuchMethodException: org.apache.hadoop.hdfs.DFSClient.decryptEncryptedDataEncryptionKey(org.apache.hadoop.fs.FileEncryptionInfo) at java.lang.Class.getDeclaredMethod(Class.java:2130) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelperWithoutHDFS12396(FanOutOneBlockAsyncDFSOutputSaslHelper.java:182) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.createTransparentCryptoHelper(FanOutOneBlockAsyncDFSOutputSaslHelper.java:241) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper.(FanOutOneBlockAsyncDFSOutputSaslHelper.java:252) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.wal.AsyncFSWALProvider.load(AsyncFSWALProvider.java:140) at org.apache.hadoop.hbase.wal.WALFactory.getProviderClass(WALFactory.java:135) at org.apache.hadoop.hbase.wal.WALFactory.getProvider(WALFactory.java:175) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:202) at org.apache.hadoop.hbase.wal.WALFactory.(WALFactory.java:182) at org.apache.hadoop.hbase.master.region.MasterRegion.create(MasterRegion.java:339) at org.apache.hadoop.hbase.master.region.MasterRegionFactory.create(MasterRegionFactory.java:104) at org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:855) at org.apache.hadoop.hbase.master.HMaster.startActiveMasterManager(HMaster.java:2193) at org.apache.hadoop.hbase.master.HMaster.lambda$run$0(HMaster.java:528) at java.lang.Thread.run(Thread.java:750) 2023-07-24 12:14:54,933 INFO [master/jenkins-hbase6:0:becomeActiveMaster] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 12:14:54,967 INFO [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(7693): Creating {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='master:store', {NAME => 'proc', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, under table dir hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/MasterData/data/master/store-tmp 2023-07-24 12:14:55,000 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:14:55,001 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-24 12:14:55,001 INFO [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 12:14:55,001 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 12:14:55,001 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-24 12:14:55,001 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 12:14:55,001 INFO [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 12:14:55,001 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-24 12:14:55,003 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] region.MasterRegion(191): WALDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/MasterData/WALs/jenkins-hbase6.apache.org,44983,1690200893288 2023-07-24 12:14:55,021 INFO [master/jenkins-hbase6:0:becomeActiveMaster] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase6.apache.org%2C44983%2C1690200893288, suffix=, logDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/MasterData/WALs/jenkins-hbase6.apache.org,44983,1690200893288, archiveDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/MasterData/oldWALs, maxLogs=10 2023-07-24 12:14:55,079 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35821,DS-9a5cb7b9-3c18-456e-b03c-b506d2a6d9f9,DISK] 2023-07-24 12:14:55,079 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44861,DS-aa809477-162f-4591-b20e-2f3db95c540a,DISK] 2023-07-24 12:14:55,079 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46387,DS-8e7af3e2-677f-457f-972b-9a9656ca819c,DISK] 2023-07-24 12:14:55,087 DEBUG [RS-EventLoopGroup-5-2] asyncfs.ProtobufDecoder(123): Hadoop 3.2 and below use unshaded protobuf. java.lang.ClassNotFoundException: org.apache.hadoop.thirdparty.protobuf.MessageLite at java.net.URLClassLoader.findClass(URLClassLoader.java:387) at java.lang.ClassLoader.loadClass(ClassLoader.java:418) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:352) at java.lang.ClassLoader.loadClass(ClassLoader.java:351) at java.lang.Class.forName0(Native Method) at java.lang.Class.forName(Class.java:264) at org.apache.hadoop.hbase.io.asyncfs.ProtobufDecoder.(ProtobufDecoder.java:118) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.processWriteBlockResponse(FanOutOneBlockAsyncDFSOutputHelper.java:340) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$100(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$4.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:424) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:557) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.addListener(DefaultPromise.java:185) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.initialize(FanOutOneBlockAsyncDFSOutputHelper.java:418) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper.access$300(FanOutOneBlockAsyncDFSOutputHelper.java:112) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:476) at org.apache.hadoop.hbase.io.asyncfs.FanOutOneBlockAsyncDFSOutputHelper$5.operationComplete(FanOutOneBlockAsyncDFSOutputHelper.java:471) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListener0(DefaultPromise.java:590) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners0(DefaultPromise.java:583) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListenersNow(DefaultPromise.java:559) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.notifyListeners(DefaultPromise.java:492) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setValue0(DefaultPromise.java:636) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.setSuccess0(DefaultPromise.java:625) at org.apache.hbase.thirdparty.io.netty.util.concurrent.DefaultPromise.trySuccess(DefaultPromise.java:105) at org.apache.hbase.thirdparty.io.netty.channel.DefaultChannelPromise.trySuccess(DefaultChannelPromise.java:84) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.fulfillConnectPromise(AbstractEpollChannel.java:653) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.finishConnect(AbstractEpollChannel.java:691) at org.apache.hbase.thirdparty.io.netty.channel.epoll.AbstractEpollChannel$AbstractEpollUnsafe.epollOutReady(AbstractEpollChannel.java:567) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:489) at org.apache.hbase.thirdparty.io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:397) at org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) at org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) at org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) at java.lang.Thread.run(Thread.java:750) 2023-07-24 12:14:55,164 INFO [master/jenkins-hbase6:0:becomeActiveMaster] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/MasterData/WALs/jenkins-hbase6.apache.org,44983,1690200893288/jenkins-hbase6.apache.org%2C44983%2C1690200893288.1690200895031 2023-07-24 12:14:55,164 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46387,DS-8e7af3e2-677f-457f-972b-9a9656ca819c,DISK], DatanodeInfoWithStorage[127.0.0.1:35821,DS-9a5cb7b9-3c18-456e-b03c-b506d2a6d9f9,DISK], DatanodeInfoWithStorage[127.0.0.1:44861,DS-aa809477-162f-4591-b20e-2f3db95c540a,DISK]] 2023-07-24 12:14:55,165 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(7854): Opening region: {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:14:55,165 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(866): Instantiated master:store,,1.1595e783b53d99cd5eef43b6debb2682.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:14:55,168 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(7894): checking encryption for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 12:14:55,169 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(7897): checking classloading for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 12:14:55,226 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family proc of region 1595e783b53d99cd5eef43b6debb2682 2023-07-24 12:14:55,234 DEBUG [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc 2023-07-24 12:14:55,259 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1595e783b53d99cd5eef43b6debb2682 columnFamilyName proc 2023-07-24 12:14:55,271 INFO [StoreOpener-1595e783b53d99cd5eef43b6debb2682-1] regionserver.HStore(310): Store=1595e783b53d99cd5eef43b6debb2682/proc, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:14:55,276 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-24 12:14:55,277 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682 2023-07-24 12:14:55,291 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1055): writing seq id for 1595e783b53d99cd5eef43b6debb2682 2023-07-24 12:14:55,295 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 12:14:55,296 INFO [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(1072): Opened 1595e783b53d99cd5eef43b6debb2682; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=75463704, jitterRate=0.12449681758880615}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 12:14:55,296 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] regionserver.HRegion(965): Region open journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-24 12:14:55,297 INFO [master/jenkins-hbase6:0:becomeActiveMaster] region.MasterRegionFlusherAndCompactor(122): Constructor flushSize=134217728, flushPerChanges=1000000, flushIntervalMs=900000, compactMin=4 2023-07-24 12:14:55,316 INFO [master/jenkins-hbase6:0:becomeActiveMaster] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:14:55,316 INFO [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:14:55,318 INFO [master/jenkins-hbase6:0:becomeActiveMaster] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:14:55,320 INFO [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:14:55,352 INFO [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 31 msec 2023-07-24 12:14:55,352 INFO [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.RemoteProcedureDispatcher(96): Instantiated, coreThreads=3 (allowCoreThreadTimeOut=true), queueMaxSize=32, operationDelay=150 2023-07-24 12:14:55,376 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] assignment.AssignmentManager(253): hbase:meta replica znodes: [] 2023-07-24 12:14:55,381 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.RegionServerTracker(124): Starting RegionServerTracker; 0 have existing ServerCrashProcedures, 0 possibly 'live' servers, and 0 'splitting'. 2023-07-24 12:14:55,404 INFO [master/jenkins-hbase6:0:becomeActiveMaster] balancer.BaseLoadBalancer(1082): slop=0.001, systemTablesOnMaster=false 2023-07-24 12:14:55,408 INFO [master/jenkins-hbase6:0:becomeActiveMaster] balancer.StochasticLoadBalancer(253): Loaded config; maxSteps=1000000, runMaxSteps=false, stepsPerRegion=800, maxRunningTime=30000, isByTable=false, CostFunctions=[RegionCountSkewCostFunction, PrimaryRegionCountSkewCostFunction, MoveCostFunction, ServerLocalityCostFunction, RackLocalityCostFunction, TableSkewCostFunction, RegionReplicaHostCostFunction, RegionReplicaRackCostFunction, ReadRequestCostFunction, WriteRequestCostFunction, MemStoreSizeCostFunction, StoreFileCostFunction] , sum of multiplier of cost functions = 0.0 etc. 2023-07-24 12:14:55,409 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/balancer 2023-07-24 12:14:55,414 INFO [master/jenkins-hbase6:0:becomeActiveMaster] normalizer.RegionNormalizerWorker(118): Normalizer rate limit set to unlimited 2023-07-24 12:14:55,418 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/normalizer 2023-07-24 12:14:55,421 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 12:14:55,422 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/split 2023-07-24 12:14:55,423 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/switch/merge 2023-07-24 12:14:55,435 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/snapshot-cleanup 2023-07-24 12:14:55,439 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 12:14:55,439 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 12:14:55,439 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 12:14:55,440 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 12:14:55,439 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/running 2023-07-24 12:14:55,440 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.HMaster(744): Active/primary master=jenkins-hbase6.apache.org,44983,1690200893288, sessionid=0x100becce0d60000, setting cluster-up flag (Was=false) 2023-07-24 12:14:55,456 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 12:14:55,462 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/flush-table-proc/acquired, /hbase/flush-table-proc/reached, /hbase/flush-table-proc/abort 2023-07-24 12:14:55,464 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase6.apache.org,44983,1690200893288 2023-07-24 12:14:55,469 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 12:14:55,480 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure.ZKProcedureUtil(251): Clearing all znodes /hbase/online-snapshot/acquired, /hbase/online-snapshot/reached, /hbase/online-snapshot/abort 2023-07-24 12:14:55,481 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure.ZKProcedureCoordinator(245): Starting controller for procedure member=jenkins-hbase6.apache.org,44983,1690200893288 2023-07-24 12:14:55,483 WARN [master/jenkins-hbase6:0:becomeActiveMaster] snapshot.SnapshotManager(302): Couldn't delete working snapshot directory: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.hbase-snapshot/.tmp 2023-07-24 12:14:55,495 INFO [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(951): ClusterId : 8a7413ce-6df9-498a-a28b-69c904dece7d 2023-07-24 12:14:55,495 INFO [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(951): ClusterId : 8a7413ce-6df9-498a-a28b-69c904dece7d 2023-07-24 12:14:55,495 INFO [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer(951): ClusterId : 8a7413ce-6df9-498a-a28b-69c904dece7d 2023-07-24 12:14:55,501 DEBUG [RS:1;jenkins-hbase6:34575] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 12:14:55,501 DEBUG [RS:2;jenkins-hbase6:36983] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 12:14:55,501 DEBUG [RS:0;jenkins-hbase6:41789] procedure.RegionServerProcedureManagerHost(43): Procedure flush-table-proc initializing 2023-07-24 12:14:55,507 DEBUG [RS:2;jenkins-hbase6:36983] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 12:14:55,507 DEBUG [RS:0;jenkins-hbase6:41789] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 12:14:55,507 DEBUG [RS:1;jenkins-hbase6:34575] procedure.RegionServerProcedureManagerHost(45): Procedure flush-table-proc initialized 2023-07-24 12:14:55,507 DEBUG [RS:0;jenkins-hbase6:41789] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 12:14:55,507 DEBUG [RS:2;jenkins-hbase6:36983] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 12:14:55,507 DEBUG [RS:1;jenkins-hbase6:34575] procedure.RegionServerProcedureManagerHost(43): Procedure online-snapshot initializing 2023-07-24 12:14:55,511 DEBUG [RS:0;jenkins-hbase6:41789] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 12:14:55,513 DEBUG [RS:1;jenkins-hbase6:34575] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 12:14:55,513 DEBUG [RS:2;jenkins-hbase6:36983] procedure.RegionServerProcedureManagerHost(45): Procedure online-snapshot initialized 2023-07-24 12:14:55,513 DEBUG [RS:0;jenkins-hbase6:41789] zookeeper.ReadOnlyZKClient(139): Connect 0x16efbeef to 127.0.0.1:58848 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 12:14:55,515 DEBUG [RS:2;jenkins-hbase6:36983] zookeeper.ReadOnlyZKClient(139): Connect 0x168a8e12 to 127.0.0.1:58848 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 12:14:55,515 DEBUG [RS:1;jenkins-hbase6:34575] zookeeper.ReadOnlyZKClient(139): Connect 0x5c6607c3 to 127.0.0.1:58848 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 12:14:55,530 DEBUG [RS:0;jenkins-hbase6:41789] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@18285242, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-24 12:14:55,531 DEBUG [RS:2;jenkins-hbase6:36983] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@184d62b3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-24 12:14:55,531 DEBUG [RS:1;jenkins-hbase6:34575] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@52a1f23f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-24 12:14:55,531 DEBUG [RS:0;jenkins-hbase6:41789] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c17d59f, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase6.apache.org/172.31.2.20:0 2023-07-24 12:14:55,531 DEBUG [RS:2;jenkins-hbase6:36983] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@1c72e562, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase6.apache.org/172.31.2.20:0 2023-07-24 12:14:55,532 DEBUG [RS:1;jenkins-hbase6:34575] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@569e6cb3, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase6.apache.org/172.31.2.20:0 2023-07-24 12:14:55,562 DEBUG [RS:0;jenkins-hbase6:41789] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:0;jenkins-hbase6:41789 2023-07-24 12:14:55,564 DEBUG [RS:2;jenkins-hbase6:36983] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:2;jenkins-hbase6:36983 2023-07-24 12:14:55,566 INFO [RS:0;jenkins-hbase6:41789] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 12:14:55,566 INFO [RS:2;jenkins-hbase6:36983] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 12:14:55,567 DEBUG [RS:1;jenkins-hbase6:34575] regionserver.ShutdownHook(81): Installed shutdown hook thread: Shutdownhook:RS:1;jenkins-hbase6:34575 2023-07-24 12:14:55,567 INFO [RS:0;jenkins-hbase6:41789] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 12:14:55,567 INFO [RS:1;jenkins-hbase6:34575] regionserver.RegionServerCoprocessorHost(66): System coprocessor loading is enabled 2023-07-24 12:14:55,567 DEBUG [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 12:14:55,567 INFO [RS:2;jenkins-hbase6:36983] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 12:14:55,567 INFO [RS:1;jenkins-hbase6:34575] regionserver.RegionServerCoprocessorHost(67): Table coprocessor loading is enabled 2023-07-24 12:14:55,567 DEBUG [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 12:14:55,567 DEBUG [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(1022): About to register with Master. 2023-07-24 12:14:55,571 INFO [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase6.apache.org,44983,1690200893288 with isa=jenkins-hbase6.apache.org/172.31.2.20:36983, startcode=1690200894572 2023-07-24 12:14:55,571 INFO [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase6.apache.org,44983,1690200893288 with isa=jenkins-hbase6.apache.org/172.31.2.20:41789, startcode=1690200894488 2023-07-24 12:14:55,571 INFO [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase6.apache.org,44983,1690200893288 with isa=jenkins-hbase6.apache.org/172.31.2.20:34575, startcode=1690200894532 2023-07-24 12:14:55,591 DEBUG [RS:0;jenkins-hbase6:41789] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 12:14:55,591 DEBUG [RS:1;jenkins-hbase6:34575] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 12:14:55,591 DEBUG [RS:2;jenkins-hbase6:36983] ipc.RpcConnection(124): Using SIMPLE authentication for service=RegionServerStatusService, sasl=false 2023-07-24 12:14:55,642 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT; InitMetaProcedure table=hbase:meta 2023-07-24 12:14:55,654 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_OPEN_REGION-master/jenkins-hbase6:0, corePoolSize=5, maxPoolSize=5 2023-07-24 12:14:55,654 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_CLOSE_REGION-master/jenkins-hbase6:0, corePoolSize=5, maxPoolSize=5 2023-07-24 12:14:55,654 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SERVER_OPERATIONS-master/jenkins-hbase6:0, corePoolSize=5, maxPoolSize=5 2023-07-24 12:14:55,654 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_META_SERVER_OPERATIONS-master/jenkins-hbase6:0, corePoolSize=5, maxPoolSize=5 2023-07-24 12:14:55,655 INFO [RS-EventLoopGroup-1-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:37637, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.0 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 12:14:55,655 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=M_LOG_REPLAY_OPS-master/jenkins-hbase6:0, corePoolSize=10, maxPoolSize=10 2023-07-24 12:14:55,655 INFO [RS-EventLoopGroup-1-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:38365, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.2 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 12:14:55,655 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_SNAPSHOT_OPERATIONS-master/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,655 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:42271, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins.hfs.1 (auth:SIMPLE), service=RegionServerStatusService 2023-07-24 12:14:55,656 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_MERGE_OPERATIONS-master/jenkins-hbase6:0, corePoolSize=2, maxPoolSize=2 2023-07-24 12:14:55,656 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] executor.ExecutorService(93): Starting executor service name=MASTER_TABLE_OPERATIONS-master/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,658 INFO [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200925658 2023-07-24 12:14:55,660 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.DirScanPool(70): log_cleaner Cleaner pool size is 1 2023-07-24 12:14:55,666 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44983] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 12:14:55,667 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_WRITE_FS_LAYOUT, locked=true; InitMetaProcedure table=hbase:meta 2023-07-24 12:14:55,667 INFO [PEWorker-1] procedure.InitMetaProcedure(71): BOOTSTRAP: creating hbase:meta region 2023-07-24 12:14:55,673 INFO [PEWorker-1] util.FSTableDescriptors(128): Creating new hbase:meta table descriptor 'hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-24 12:14:55,673 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveLogCleaner 2023-07-24 12:14:55,676 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44983] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 12:14:55,678 DEBUG [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44983] ipc.MetricsHBaseServer(134): Unknown exception type org.apache.hadoop.hbase.ipc.ServerNotRunningYetException: Server is not running yet at org.apache.hadoop.hbase.master.HMaster.checkServiceStarted(HMaster.java:2832) at org.apache.hadoop.hbase.master.MasterRpcServices.regionServerStartup(MasterRpcServices.java:579) at org.apache.hadoop.hbase.shaded.protobuf.generated.RegionServerStatusProtos$RegionServerStatusService$2.callBlockingMethod(RegionServerStatusProtos.java:15952) at org.apache.hadoop.hbase.ipc.RpcServer.call(RpcServer.java:387) at org.apache.hadoop.hbase.ipc.CallRunner.run(CallRunner.java:132) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:369) at org.apache.hadoop.hbase.ipc.RpcExecutor$Handler.run(RpcExecutor.java:349) 2023-07-24 12:14:55,685 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.replication.master.ReplicationLogCleaner 2023-07-24 12:14:55,686 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreWALCleaner 2023-07-24 12:14:55,686 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveProcedureWALCleaner 2023-07-24 12:14:55,686 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.LogCleaner(148): Creating 1 old WALs cleaner threads 2023-07-24 12:14:55,688 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=LogsCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,689 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.DirScanPool(70): hfile_cleaner Cleaner pool size is 2 2023-07-24 12:14:55,691 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveMasterLocalStoreHFileCleaner 2023-07-24 12:14:55,691 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.HFileLinkCleaner 2023-07-24 12:14:55,695 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.snapshot.SnapshotHFileCleaner 2023-07-24 12:14:55,695 INFO [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.CleanerChore(175): Initialize cleaner=org.apache.hadoop.hbase.master.cleaner.TimeToLiveHFileCleaner 2023-07-24 12:14:55,699 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.HFileCleaner(242): Starting for large file=Thread[master/jenkins-hbase6:0:becomeActiveMaster-HFileCleaner.large.0-1690200895697,5,FailOnTimeoutGroup] 2023-07-24 12:14:55,699 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] cleaner.HFileCleaner(257): Starting for small files=Thread[master/jenkins-hbase6:0:becomeActiveMaster-HFileCleaner.small.0-1690200895699,5,FailOnTimeoutGroup] 2023-07-24 12:14:55,700 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HFileCleaner, period=600000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,700 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.HMaster(1461): Reopening regions with very high storeFileRefCount is disabled. Provide threshold value > 0 for hbase.regions.recovery.store.file.ref.count to enable it. 2023-07-24 12:14:55,702 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=ReplicationBarrierCleaner, period=43200000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,702 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=SnapshotCleaner, period=1800000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,705 DEBUG [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer(2830): Master is not running yet 2023-07-24 12:14:55,705 WARN [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-24 12:14:55,706 DEBUG [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(2830): Master is not running yet 2023-07-24 12:14:55,706 DEBUG [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(2830): Master is not running yet 2023-07-24 12:14:55,706 WARN [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-24 12:14:55,706 WARN [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(1030): reportForDuty failed; sleeping 100 ms and then retrying. 2023-07-24 12:14:55,722 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-24 12:14:55,723 INFO [PEWorker-1] util.FSTableDescriptors(135): Updated hbase:meta table descriptor to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/.tabledesc/.tableinfo.0000000001 2023-07-24 12:14:55,723 INFO [PEWorker-1] regionserver.HRegion(7675): creating {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:meta', {TABLE_ATTRIBUTES => {IS_META => 'true', coprocessor$1 => '|org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint|536870911|'}}, {NAME => 'info', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, {NAME => 'rep_barrier', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '2147483647', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'table', BLOOMFILTER => 'NONE', IN_MEMORY => 'true', VERSIONS => '3', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef 2023-07-24 12:14:55,745 DEBUG [PEWorker-1] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:14:55,747 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-24 12:14:55,750 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/info 2023-07-24 12:14:55,751 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-24 12:14:55,752 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:14:55,752 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-24 12:14:55,754 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/rep_barrier 2023-07-24 12:14:55,755 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-24 12:14:55,756 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:14:55,756 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-24 12:14:55,758 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/table 2023-07-24 12:14:55,758 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-24 12:14:55,759 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:14:55,761 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740 2023-07-24 12:14:55,762 DEBUG [PEWorker-1] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740 2023-07-24 12:14:55,766 DEBUG [PEWorker-1] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-24 12:14:55,768 DEBUG [PEWorker-1] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-24 12:14:55,772 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 12:14:55,773 INFO [PEWorker-1] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62137329, jitterRate=-0.07408164441585541}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-24 12:14:55,773 DEBUG [PEWorker-1] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-24 12:14:55,773 DEBUG [PEWorker-1] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-24 12:14:55,773 INFO [PEWorker-1] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-24 12:14:55,773 DEBUG [PEWorker-1] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-24 12:14:55,773 DEBUG [PEWorker-1] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-24 12:14:55,773 DEBUG [PEWorker-1] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-24 12:14:55,775 INFO [PEWorker-1] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-24 12:14:55,775 DEBUG [PEWorker-1] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-24 12:14:55,780 DEBUG [PEWorker-1] procedure.InitMetaProcedure(92): Execute pid=1, state=RUNNABLE:INIT_META_ASSIGN_META, locked=true; InitMetaProcedure table=hbase:meta 2023-07-24 12:14:55,780 INFO [PEWorker-1] procedure.InitMetaProcedure(103): Going to assign meta 2023-07-24 12:14:55,789 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN}] 2023-07-24 12:14:55,802 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN 2023-07-24 12:14:55,804 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=2, ppid=1, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-24 12:14:55,806 INFO [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase6.apache.org,44983,1690200893288 with isa=jenkins-hbase6.apache.org/172.31.2.20:36983, startcode=1690200894572 2023-07-24 12:14:55,807 INFO [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase6.apache.org,44983,1690200893288 with isa=jenkins-hbase6.apache.org/172.31.2.20:41789, startcode=1690200894488 2023-07-24 12:14:55,807 INFO [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(2811): reportForDuty to master=jenkins-hbase6.apache.org,44983,1690200893288 with isa=jenkins-hbase6.apache.org/172.31.2.20:34575, startcode=1690200894532 2023-07-24 12:14:55,812 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44983] master.ServerManager(394): Registering regionserver=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:55,817 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44983] master.ServerManager(394): Registering regionserver=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:55,817 DEBUG [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef 2023-07-24 12:14:55,818 INFO [RpcServer.priority.RWQ.Fifo.write.handler=0,queue=0,port=44983] master.ServerManager(394): Registering regionserver=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:55,818 DEBUG [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33821 2023-07-24 12:14:55,818 DEBUG [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-24 12:14:55,818 DEBUG [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef 2023-07-24 12:14:55,818 DEBUG [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33821 2023-07-24 12:14:55,819 DEBUG [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-24 12:14:55,819 DEBUG [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(1595): Config from master: hbase.rootdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef 2023-07-24 12:14:55,819 DEBUG [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(1595): Config from master: fs.defaultFS=hdfs://localhost:33821 2023-07-24 12:14:55,819 DEBUG [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(1595): Config from master: hbase.master.info.port=-1 2023-07-24 12:14:55,830 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 12:14:55,831 DEBUG [RS:1;jenkins-hbase6:34575] zookeeper.ZKUtil(162): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:55,831 DEBUG [RS:0;jenkins-hbase6:41789] zookeeper.ZKUtil(162): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:55,831 WARN [RS:1;jenkins-hbase6:34575] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 12:14:55,831 DEBUG [RS:2;jenkins-hbase6:36983] zookeeper.ZKUtil(162): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:55,832 INFO [RS:1;jenkins-hbase6:34575] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 12:14:55,832 WARN [RS:2;jenkins-hbase6:36983] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 12:14:55,831 WARN [RS:0;jenkins-hbase6:41789] hbase.ZNodeClearer(69): Environment variable HBASE_ZNODE_FILE not set; znodes will not be cleared on crash by start scripts (Longer MTTR!) 2023-07-24 12:14:55,832 DEBUG [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/WALs/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:55,832 INFO [RS:2;jenkins-hbase6:36983] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 12:14:55,832 INFO [RS:0;jenkins-hbase6:41789] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 12:14:55,833 DEBUG [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/WALs/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:55,833 DEBUG [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(1948): logDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/WALs/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:55,833 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase6.apache.org,34575,1690200894532] 2023-07-24 12:14:55,833 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase6.apache.org,41789,1690200894488] 2023-07-24 12:14:55,833 INFO [RegionServerTracker-0] master.RegionServerTracker(190): RegionServer ephemeral node created, adding [jenkins-hbase6.apache.org,36983,1690200894572] 2023-07-24 12:14:55,845 DEBUG [RS:1;jenkins-hbase6:34575] zookeeper.ZKUtil(162): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:55,845 DEBUG [RS:0;jenkins-hbase6:41789] zookeeper.ZKUtil(162): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:55,845 DEBUG [RS:2;jenkins-hbase6:36983] zookeeper.ZKUtil(162): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:55,845 DEBUG [RS:1;jenkins-hbase6:34575] zookeeper.ZKUtil(162): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:55,845 DEBUG [RS:0;jenkins-hbase6:41789] zookeeper.ZKUtil(162): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:55,846 DEBUG [RS:2;jenkins-hbase6:36983] zookeeper.ZKUtil(162): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:55,846 DEBUG [RS:1;jenkins-hbase6:34575] zookeeper.ZKUtil(162): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:55,846 DEBUG [RS:0;jenkins-hbase6:41789] zookeeper.ZKUtil(162): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:55,846 DEBUG [RS:2;jenkins-hbase6:36983] zookeeper.ZKUtil(162): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/rs/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:55,856 DEBUG [RS:1;jenkins-hbase6:34575] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 12:14:55,856 DEBUG [RS:0;jenkins-hbase6:41789] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 12:14:55,856 DEBUG [RS:2;jenkins-hbase6:36983] regionserver.Replication(139): Replication stats-in-log period=300 seconds 2023-07-24 12:14:55,866 INFO [RS:2;jenkins-hbase6:36983] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 12:14:55,866 INFO [RS:1;jenkins-hbase6:34575] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 12:14:55,866 INFO [RS:0;jenkins-hbase6:41789] regionserver.MetricsRegionServerWrapperImpl(154): Computing regionserver metrics every 5000 milliseconds 2023-07-24 12:14:55,886 INFO [RS:2;jenkins-hbase6:36983] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 12:14:55,886 INFO [RS:0;jenkins-hbase6:41789] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 12:14:55,886 INFO [RS:1;jenkins-hbase6:34575] regionserver.MemStoreFlusher(125): globalMemStoreLimit=782.4 M, globalMemStoreLimitLowMark=743.3 M, Offheap=false 2023-07-24 12:14:55,892 INFO [RS:2;jenkins-hbase6:36983] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 12:14:55,892 INFO [RS:1;jenkins-hbase6:34575] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 12:14:55,892 INFO [RS:2;jenkins-hbase6:36983] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,892 INFO [RS:0;jenkins-hbase6:41789] throttle.PressureAwareCompactionThroughputController(131): Compaction throughput configurations, higher bound: 100.00 MB/second, lower bound 50.00 MB/second, off peak: unlimited, tuning period: 60000 ms 2023-07-24 12:14:55,892 INFO [RS:1;jenkins-hbase6:34575] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,893 INFO [RS:0;jenkins-hbase6:41789] hbase.ChoreService(166): Chore ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,893 INFO [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 12:14:55,895 INFO [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 12:14:55,895 INFO [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer$CompactionChecker(1837): CompactionChecker runs every PT1S 2023-07-24 12:14:55,903 INFO [RS:1;jenkins-hbase6:34575] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,903 INFO [RS:0;jenkins-hbase6:41789] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,903 INFO [RS:2;jenkins-hbase6:36983] hbase.ChoreService(166): Chore ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,903 DEBUG [RS:1;jenkins-hbase6:34575] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,903 DEBUG [RS:0;jenkins-hbase6:41789] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,904 DEBUG [RS:2;jenkins-hbase6:36983] executor.ExecutorService(93): Starting executor service name=RS_OPEN_REGION-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,904 DEBUG [RS:1;jenkins-hbase6:34575] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,904 DEBUG [RS:2;jenkins-hbase6:36983] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,904 DEBUG [RS:1;jenkins-hbase6:34575] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,904 DEBUG [RS:0;jenkins-hbase6:41789] executor.ExecutorService(93): Starting executor service name=RS_OPEN_META-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,904 DEBUG [RS:1;jenkins-hbase6:34575] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,904 DEBUG [RS:2;jenkins-hbase6:36983] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,904 DEBUG [RS:1;jenkins-hbase6:34575] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,904 DEBUG [RS:2;jenkins-hbase6:36983] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,904 DEBUG [RS:1;jenkins-hbase6:34575] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase6:0, corePoolSize=2, maxPoolSize=2 2023-07-24 12:14:55,904 DEBUG [RS:2;jenkins-hbase6:36983] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,904 DEBUG [RS:1;jenkins-hbase6:34575] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,905 DEBUG [RS:2;jenkins-hbase6:36983] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase6:0, corePoolSize=2, maxPoolSize=2 2023-07-24 12:14:55,904 DEBUG [RS:0;jenkins-hbase6:41789] executor.ExecutorService(93): Starting executor service name=RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,905 DEBUG [RS:2;jenkins-hbase6:36983] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,905 DEBUG [RS:1;jenkins-hbase6:34575] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,905 DEBUG [RS:2;jenkins-hbase6:36983] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,905 DEBUG [RS:1;jenkins-hbase6:34575] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,905 DEBUG [RS:2;jenkins-hbase6:36983] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,905 DEBUG [RS:1;jenkins-hbase6:34575] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,905 DEBUG [RS:2;jenkins-hbase6:36983] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,905 DEBUG [RS:0;jenkins-hbase6:41789] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_REGION-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,905 DEBUG [RS:0;jenkins-hbase6:41789] executor.ExecutorService(93): Starting executor service name=RS_CLOSE_META-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,906 DEBUG [RS:0;jenkins-hbase6:41789] executor.ExecutorService(93): Starting executor service name=RS_LOG_REPLAY_OPS-regionserver/jenkins-hbase6:0, corePoolSize=2, maxPoolSize=2 2023-07-24 12:14:55,906 DEBUG [RS:0;jenkins-hbase6:41789] executor.ExecutorService(93): Starting executor service name=RS_COMPACTED_FILES_DISCHARGER-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,906 DEBUG [RS:0;jenkins-hbase6:41789] executor.ExecutorService(93): Starting executor service name=RS_REGION_REPLICA_FLUSH_OPS-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,906 DEBUG [RS:0;jenkins-hbase6:41789] executor.ExecutorService(93): Starting executor service name=RS_REFRESH_PEER-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,906 DEBUG [RS:0;jenkins-hbase6:41789] executor.ExecutorService(93): Starting executor service name=RS_SWITCH_RPC_THROTTLE-regionserver/jenkins-hbase6:0, corePoolSize=1, maxPoolSize=1 2023-07-24 12:14:55,908 INFO [RS:2;jenkins-hbase6:36983] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,908 INFO [RS:2;jenkins-hbase6:36983] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,908 INFO [RS:1;jenkins-hbase6:34575] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,908 INFO [RS:2;jenkins-hbase6:36983] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,909 INFO [RS:1;jenkins-hbase6:34575] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,909 INFO [RS:1;jenkins-hbase6:34575] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,910 INFO [RS:0;jenkins-hbase6:41789] hbase.ChoreService(166): Chore ScheduledChore name=CompactionChecker, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,910 INFO [RS:0;jenkins-hbase6:41789] hbase.ChoreService(166): Chore ScheduledChore name=MemstoreFlusherChore, period=1000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,910 INFO [RS:0;jenkins-hbase6:41789] hbase.ChoreService(166): Chore ScheduledChore name=nonceCleaner, period=360000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,927 INFO [RS:0;jenkins-hbase6:41789] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 12:14:55,927 INFO [RS:1;jenkins-hbase6:34575] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 12:14:55,927 INFO [RS:2;jenkins-hbase6:36983] regionserver.HeapMemoryManager(209): Starting, tuneOn=false 2023-07-24 12:14:55,930 INFO [RS:0;jenkins-hbase6:41789] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase6.apache.org,41789,1690200894488-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,930 INFO [RS:1;jenkins-hbase6:34575] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase6.apache.org,34575,1690200894532-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,930 INFO [RS:2;jenkins-hbase6:36983] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase6.apache.org,36983,1690200894572-HeapMemoryTunerChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:55,956 DEBUG [jenkins-hbase6:44983] assignment.AssignmentManager(2176): Processing assignQueue; systemServersCount=3, allServersCount=3 2023-07-24 12:14:55,958 INFO [RS:0;jenkins-hbase6:41789] regionserver.Replication(203): jenkins-hbase6.apache.org,41789,1690200894488 started 2023-07-24 12:14:55,958 INFO [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(1637): Serving as jenkins-hbase6.apache.org,41789,1690200894488, RpcServer on jenkins-hbase6.apache.org/172.31.2.20:41789, sessionid=0x100becce0d60001 2023-07-24 12:14:55,958 DEBUG [RS:0;jenkins-hbase6:41789] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 12:14:55,958 DEBUG [RS:0;jenkins-hbase6:41789] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:55,958 DEBUG [RS:0;jenkins-hbase6:41789] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase6.apache.org,41789,1690200894488' 2023-07-24 12:14:55,958 DEBUG [RS:0;jenkins-hbase6:41789] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:55,959 DEBUG [RS:0;jenkins-hbase6:41789] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:55,959 INFO [RS:2;jenkins-hbase6:36983] regionserver.Replication(203): jenkins-hbase6.apache.org,36983,1690200894572 started 2023-07-24 12:14:55,959 INFO [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer(1637): Serving as jenkins-hbase6.apache.org,36983,1690200894572, RpcServer on jenkins-hbase6.apache.org/172.31.2.20:36983, sessionid=0x100becce0d60003 2023-07-24 12:14:55,959 DEBUG [RS:2;jenkins-hbase6:36983] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 12:14:55,959 DEBUG [RS:0;jenkins-hbase6:41789] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 12:14:55,959 DEBUG [RS:2;jenkins-hbase6:36983] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:55,960 DEBUG [RS:0;jenkins-hbase6:41789] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 12:14:55,960 DEBUG [RS:2;jenkins-hbase6:36983] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase6.apache.org,36983,1690200894572' 2023-07-24 12:14:55,961 DEBUG [RS:2;jenkins-hbase6:36983] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:55,961 DEBUG [RS:0;jenkins-hbase6:41789] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:55,961 DEBUG [RS:0;jenkins-hbase6:41789] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase6.apache.org,41789,1690200894488' 2023-07-24 12:14:55,961 DEBUG [RS:0;jenkins-hbase6:41789] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 12:14:55,962 DEBUG [jenkins-hbase6:44983] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase6.apache.org=0} racks are {/default-rack=0} 2023-07-24 12:14:55,962 DEBUG [RS:0;jenkins-hbase6:41789] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 12:14:55,962 DEBUG [RS:2;jenkins-hbase6:36983] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:55,963 DEBUG [RS:2;jenkins-hbase6:36983] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 12:14:55,963 DEBUG [RS:0;jenkins-hbase6:41789] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 12:14:55,963 DEBUG [RS:2;jenkins-hbase6:36983] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 12:14:55,963 INFO [RS:0;jenkins-hbase6:41789] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 12:14:55,963 DEBUG [RS:2;jenkins-hbase6:36983] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:55,968 DEBUG [RS:2;jenkins-hbase6:36983] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase6.apache.org,36983,1690200894572' 2023-07-24 12:14:55,968 DEBUG [RS:2;jenkins-hbase6:36983] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 12:14:55,968 INFO [RS:1;jenkins-hbase6:34575] regionserver.Replication(203): jenkins-hbase6.apache.org,34575,1690200894532 started 2023-07-24 12:14:55,968 INFO [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(1637): Serving as jenkins-hbase6.apache.org,34575,1690200894532, RpcServer on jenkins-hbase6.apache.org/172.31.2.20:34575, sessionid=0x100becce0d60002 2023-07-24 12:14:55,968 INFO [RS:0;jenkins-hbase6:41789] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 12:14:55,969 DEBUG [RS:2;jenkins-hbase6:36983] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 12:14:55,969 DEBUG [RS:1;jenkins-hbase6:34575] procedure.RegionServerProcedureManagerHost(51): Procedure flush-table-proc starting 2023-07-24 12:14:55,969 DEBUG [RS:1;jenkins-hbase6:34575] flush.RegionServerFlushTableProcedureManager(106): Start region server flush procedure manager jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:55,969 DEBUG [RS:1;jenkins-hbase6:34575] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase6.apache.org,34575,1690200894532' 2023-07-24 12:14:55,969 DEBUG [RS:1;jenkins-hbase6:34575] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:55,969 DEBUG [RS:2;jenkins-hbase6:36983] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 12:14:55,969 INFO [RS:2;jenkins-hbase6:36983] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 12:14:55,970 INFO [RS:2;jenkins-hbase6:36983] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 12:14:55,971 DEBUG [RS:1;jenkins-hbase6:34575] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:55,971 DEBUG [RS:1;jenkins-hbase6:34575] procedure.RegionServerProcedureManagerHost(53): Procedure flush-table-proc started 2023-07-24 12:14:55,971 DEBUG [RS:1;jenkins-hbase6:34575] procedure.RegionServerProcedureManagerHost(51): Procedure online-snapshot starting 2023-07-24 12:14:55,971 DEBUG [RS:1;jenkins-hbase6:34575] snapshot.RegionServerSnapshotManager(126): Start Snapshot Manager jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:55,972 DEBUG [RS:1;jenkins-hbase6:34575] procedure.ZKProcedureMemberRpcs(357): Starting procedure member 'jenkins-hbase6.apache.org,34575,1690200894532' 2023-07-24 12:14:55,972 DEBUG [RS:1;jenkins-hbase6:34575] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/online-snapshot/abort' 2023-07-24 12:14:55,972 DEBUG [RS:1;jenkins-hbase6:34575] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/online-snapshot/acquired' 2023-07-24 12:14:55,973 DEBUG [RS:1;jenkins-hbase6:34575] procedure.RegionServerProcedureManagerHost(53): Procedure online-snapshot started 2023-07-24 12:14:55,973 INFO [RS:1;jenkins-hbase6:34575] quotas.RegionServerRpcQuotaManager(63): Quota support disabled 2023-07-24 12:14:55,973 INFO [RS:1;jenkins-hbase6:34575] quotas.RegionServerSpaceQuotaManager(80): Quota support disabled, not starting space quota manager. 2023-07-24 12:14:55,974 DEBUG [jenkins-hbase6:44983] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 12:14:55,975 DEBUG [jenkins-hbase6:44983] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 12:14:55,975 DEBUG [jenkins-hbase6:44983] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 12:14:55,975 DEBUG [jenkins-hbase6:44983] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 12:14:55,978 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase6.apache.org,41789,1690200894488, state=OPENING 2023-07-24 12:14:55,986 DEBUG [PEWorker-1] zookeeper.MetaTableLocator(240): hbase:meta region location doesn't exist, create it 2023-07-24 12:14:55,988 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 12:14:55,988 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-24 12:14:55,991 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=3, ppid=2, state=RUNNABLE; OpenRegionProcedure 1588230740, server=jenkins-hbase6.apache.org,41789,1690200894488}] 2023-07-24 12:14:56,080 INFO [RS:1;jenkins-hbase6:34575] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase6.apache.org%2C34575%2C1690200894532, suffix=, logDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/WALs/jenkins-hbase6.apache.org,34575,1690200894532, archiveDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/oldWALs, maxLogs=32 2023-07-24 12:14:56,080 INFO [RS:2;jenkins-hbase6:36983] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase6.apache.org%2C36983%2C1690200894572, suffix=, logDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/WALs/jenkins-hbase6.apache.org,36983,1690200894572, archiveDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/oldWALs, maxLogs=32 2023-07-24 12:14:56,080 INFO [RS:0;jenkins-hbase6:41789] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase6.apache.org%2C41789%2C1690200894488, suffix=, logDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/WALs/jenkins-hbase6.apache.org,41789,1690200894488, archiveDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/oldWALs, maxLogs=32 2023-07-24 12:14:56,111 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35821,DS-9a5cb7b9-3c18-456e-b03c-b506d2a6d9f9,DISK] 2023-07-24 12:14:56,117 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35821,DS-9a5cb7b9-3c18-456e-b03c-b506d2a6d9f9,DISK] 2023-07-24 12:14:56,117 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46387,DS-8e7af3e2-677f-457f-972b-9a9656ca819c,DISK] 2023-07-24 12:14:56,118 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46387,DS-8e7af3e2-677f-457f-972b-9a9656ca819c,DISK] 2023-07-24 12:14:56,118 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44861,DS-aa809477-162f-4591-b20e-2f3db95c540a,DISK] 2023-07-24 12:14:56,118 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44861,DS-aa809477-162f-4591-b20e-2f3db95c540a,DISK] 2023-07-24 12:14:56,132 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35821,DS-9a5cb7b9-3c18-456e-b03c-b506d2a6d9f9,DISK] 2023-07-24 12:14:56,133 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44861,DS-aa809477-162f-4591-b20e-2f3db95c540a,DISK] 2023-07-24 12:14:56,133 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46387,DS-8e7af3e2-677f-457f-972b-9a9656ca819c,DISK] 2023-07-24 12:14:56,144 INFO [RS:2;jenkins-hbase6:36983] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/WALs/jenkins-hbase6.apache.org,36983,1690200894572/jenkins-hbase6.apache.org%2C36983%2C1690200894572.1690200896087 2023-07-24 12:14:56,144 INFO [RS:1;jenkins-hbase6:34575] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/WALs/jenkins-hbase6.apache.org,34575,1690200894532/jenkins-hbase6.apache.org%2C34575%2C1690200894532.1690200896087 2023-07-24 12:14:56,145 DEBUG [RS:2;jenkins-hbase6:36983] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:44861,DS-aa809477-162f-4591-b20e-2f3db95c540a,DISK], DatanodeInfoWithStorage[127.0.0.1:46387,DS-8e7af3e2-677f-457f-972b-9a9656ca819c,DISK], DatanodeInfoWithStorage[127.0.0.1:35821,DS-9a5cb7b9-3c18-456e-b03c-b506d2a6d9f9,DISK]] 2023-07-24 12:14:56,145 DEBUG [RS:1;jenkins-hbase6:34575] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46387,DS-8e7af3e2-677f-457f-972b-9a9656ca819c,DISK], DatanodeInfoWithStorage[127.0.0.1:44861,DS-aa809477-162f-4591-b20e-2f3db95c540a,DISK], DatanodeInfoWithStorage[127.0.0.1:35821,DS-9a5cb7b9-3c18-456e-b03c-b506d2a6d9f9,DISK]] 2023-07-24 12:14:56,145 INFO [RS:0;jenkins-hbase6:41789] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/WALs/jenkins-hbase6.apache.org,41789,1690200894488/jenkins-hbase6.apache.org%2C41789%2C1690200894488.1690200896087 2023-07-24 12:14:56,145 DEBUG [RS:0;jenkins-hbase6:41789] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:35821,DS-9a5cb7b9-3c18-456e-b03c-b506d2a6d9f9,DISK], DatanodeInfoWithStorage[127.0.0.1:46387,DS-8e7af3e2-677f-457f-972b-9a9656ca819c,DISK], DatanodeInfoWithStorage[127.0.0.1:44861,DS-aa809477-162f-4591-b20e-2f3db95c540a,DISK]] 2023-07-24 12:14:56,182 DEBUG [RSProcedureDispatcher-pool-0] master.ServerManager(712): New admin connection to jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:56,185 DEBUG [RSProcedureDispatcher-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 12:14:56,188 INFO [RS-EventLoopGroup-3-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:38646, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 12:14:56,201 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open hbase:meta,,1.1588230740 2023-07-24 12:14:56,201 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] wal.WALFactory(158): Instantiating WALProvider of type class org.apache.hadoop.hbase.wal.AsyncFSWALProvider 2023-07-24 12:14:56,204 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] wal.AbstractFSWAL(489): WAL configuration: blocksize=256 MB, rollsize=128 MB, prefix=jenkins-hbase6.apache.org%2C41789%2C1690200894488.meta, suffix=.meta, logDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/WALs/jenkins-hbase6.apache.org,41789,1690200894488, archiveDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/oldWALs, maxLogs=32 2023-07-24 12:14:56,223 DEBUG [RS-EventLoopGroup-5-1] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:46387,DS-8e7af3e2-677f-457f-972b-9a9656ca819c,DISK] 2023-07-24 12:14:56,224 DEBUG [RS-EventLoopGroup-5-2] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:44861,DS-aa809477-162f-4591-b20e-2f3db95c540a,DISK] 2023-07-24 12:14:56,225 DEBUG [RS-EventLoopGroup-5-3] asyncfs.FanOutOneBlockAsyncDFSOutputSaslHelper(809): SASL client skipping handshake in unsecured configuration for addr = 127.0.0.1/127.0.0.1, datanodeId = DatanodeInfoWithStorage[127.0.0.1:35821,DS-9a5cb7b9-3c18-456e-b03c-b506d2a6d9f9,DISK] 2023-07-24 12:14:56,235 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] wal.AbstractFSWAL(806): New WAL /user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/WALs/jenkins-hbase6.apache.org,41789,1690200894488/jenkins-hbase6.apache.org%2C41789%2C1690200894488.meta.1690200896206.meta 2023-07-24 12:14:56,236 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] wal.AbstractFSWAL(887): Create new AsyncFSWAL writer with pipeline: [DatanodeInfoWithStorage[127.0.0.1:46387,DS-8e7af3e2-677f-457f-972b-9a9656ca819c,DISK], DatanodeInfoWithStorage[127.0.0.1:44861,DS-aa809477-162f-4591-b20e-2f3db95c540a,DISK], DatanodeInfoWithStorage[127.0.0.1:35821,DS-9a5cb7b9-3c18-456e-b03c-b506d2a6d9f9,DISK]] 2023-07-24 12:14:56,236 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 1588230740, NAME => 'hbase:meta,,1', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:14:56,240 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:14:56,241 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(215): Loading coprocessor class org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint with path null and priority 536870911 2023-07-24 12:14:56,256 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(8550): Registered coprocessor service: region=hbase:meta,,1 service=MultiRowMutationService 2023-07-24 12:14:56,259 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.RegionCoprocessorHost(393): Loaded coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint from HTD of hbase:meta successfully. 2023-07-24 12:14:56,263 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table meta 1588230740 2023-07-24 12:14:56,263 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated hbase:meta,,1.1588230740; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:14:56,263 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 1588230740 2023-07-24 12:14:56,263 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 1588230740 2023-07-24 12:14:56,266 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region 1588230740 2023-07-24 12:14:56,268 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/info 2023-07-24 12:14:56,268 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/info 2023-07-24 12:14:56,269 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName info 2023-07-24 12:14:56,271 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:14:56,272 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family rep_barrier of region 1588230740 2023-07-24 12:14:56,275 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/rep_barrier 2023-07-24 12:14:56,275 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/rep_barrier 2023-07-24 12:14:56,275 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName rep_barrier 2023-07-24 12:14:56,276 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/rep_barrier, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:14:56,276 INFO [StoreOpener-1588230740-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family table of region 1588230740 2023-07-24 12:14:56,278 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/table 2023-07-24 12:14:56,278 DEBUG [StoreOpener-1588230740-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/table 2023-07-24 12:14:56,279 INFO [StoreOpener-1588230740-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 1588230740 columnFamilyName table 2023-07-24 12:14:56,279 INFO [StoreOpener-1588230740-1] regionserver.HStore(310): Store=1588230740/table, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:14:56,282 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740 2023-07-24 12:14:56,285 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740 2023-07-24 12:14:56,288 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table hbase:meta descriptor;using region.getMemStoreFlushHeapSize/# of families (42.7 M)) instead. 2023-07-24 12:14:56,291 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 1588230740 2023-07-24 12:14:56,292 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 1588230740; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74978576, jitterRate=0.11726784706115723}}}, FlushLargeStoresPolicy{flushSizeLowerBound=44739242} 2023-07-24 12:14:56,292 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 1588230740: 2023-07-24 12:14:56,304 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:meta,,1.1588230740, pid=3, masterSystemTime=1690200896175 2023-07-24 12:14:56,322 DEBUG [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:meta,,1.1588230740 2023-07-24 12:14:56,322 INFO [RS_OPEN_META-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened hbase:meta,,1.1588230740 2023-07-24 12:14:56,323 INFO [PEWorker-1] zookeeper.MetaTableLocator(228): Setting hbase:meta replicaId=0 location in ZooKeeper as jenkins-hbase6.apache.org,41789,1690200894488, state=OPEN 2023-07-24 12:14:56,326 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/meta-region-server 2023-07-24 12:14:56,326 DEBUG [zk-event-processor-pool-0] master.MetaRegionLocationCache(164): Updating meta znode for path /hbase/meta-region-server: CHANGED 2023-07-24 12:14:56,330 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=3, resume processing ppid=2 2023-07-24 12:14:56,330 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=3, ppid=2, state=SUCCESS; OpenRegionProcedure 1588230740, server=jenkins-hbase6.apache.org,41789,1690200894488 in 335 msec 2023-07-24 12:14:56,335 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=2, resume processing ppid=1 2023-07-24 12:14:56,336 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=2, ppid=1, state=SUCCESS; TransitRegionStateProcedure table=hbase:meta, region=1588230740, ASSIGN in 543 msec 2023-07-24 12:14:56,341 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta in 786 msec 2023-07-24 12:14:56,341 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.HMaster(953): Wait for region servers to report in: status=null, state=RUNNING, startTime=1690200896341, completionTime=-1 2023-07-24 12:14:56,341 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.ServerManager(821): Finished waiting on RegionServer count=3; waited=0ms, expected min=3 server(s), max=3 server(s), master is running 2023-07-24 12:14:56,341 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:14:56,407 DEBUG [hconnection-0x595be136-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 12:14:56,410 INFO [RS-EventLoopGroup-3-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:38662, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 12:14:56,425 INFO [master/jenkins-hbase6:0:becomeActiveMaster] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:14:56,425 INFO [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200956425 2023-07-24 12:14:56,425 INFO [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201016425 2023-07-24 12:14:56,425 INFO [master/jenkins-hbase6:0:becomeActiveMaster] assignment.AssignmentManager(1536): Joined the cluster in 84 msec 2023-07-24 12:14:56,448 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase6.apache.org,44983,1690200893288-ClusterStatusChore, period=60000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:56,449 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase6.apache.org,44983,1690200893288-BalancerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:56,449 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase6.apache.org,44983,1690200893288-RegionNormalizerChore, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:56,452 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=CatalogJanitor-jenkins-hbase6:44983, period=300000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:56,452 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=HbckChore-, period=3600000, unit=MILLISECONDS is enabled. 2023-07-24 12:14:56,462 DEBUG [master/jenkins-hbase6:0.Chore.1] janitor.CatalogJanitor(175): 2023-07-24 12:14:56,471 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.TableNamespaceManager(92): Namespace table not found. Creating... 2023-07-24 12:14:56,472 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.HMaster(2148): Client=null/null create 'hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'} 2023-07-24 12:14:56,483 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=hbase:namespace 2023-07-24 12:14:56,486 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 12:14:56,488 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 12:14:56,510 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/hbase/namespace/f3d7ba30038785f88ca0646c63b6c13d 2023-07-24 12:14:56,512 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/hbase/namespace/f3d7ba30038785f88ca0646c63b6c13d empty. 2023-07-24 12:14:56,513 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/hbase/namespace/f3d7ba30038785f88ca0646c63b6c13d 2023-07-24 12:14:56,513 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived hbase:namespace regions 2023-07-24 12:14:56,551 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/hbase/namespace/.tabledesc/.tableinfo.0000000001 2023-07-24 12:14:56,553 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(7675): creating {ENCODED => f3d7ba30038785f88ca0646c63b6c13d, NAME => 'hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='hbase:namespace', {NAME => 'info', BLOOMFILTER => 'ROW', IN_MEMORY => 'true', VERSIONS => '10', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '8192', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp 2023-07-24 12:14:56,570 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:14:56,570 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1604): Closing f3d7ba30038785f88ca0646c63b6c13d, disabling compactions & flushes 2023-07-24 12:14:56,571 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d. 2023-07-24 12:14:56,571 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d. 2023-07-24 12:14:56,571 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d. after waiting 0 ms 2023-07-24 12:14:56,571 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d. 2023-07-24 12:14:56,571 INFO [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d. 2023-07-24 12:14:56,571 DEBUG [RegionOpenAndInit-hbase:namespace-pool-0] regionserver.HRegion(1558): Region close journal for f3d7ba30038785f88ca0646c63b6c13d: 2023-07-24 12:14:56,576 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 12:14:56,599 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690200896580"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200896580"}]},"ts":"1690200896580"} 2023-07-24 12:14:56,632 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 12:14:56,634 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 12:14:56,638 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200896634"}]},"ts":"1690200896634"} 2023-07-24 12:14:56,642 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLING in hbase:meta 2023-07-24 12:14:56,646 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase6.apache.org=0} racks are {/default-rack=0} 2023-07-24 12:14:56,648 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 12:14:56,648 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 12:14:56,648 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 12:14:56,648 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 12:14:56,650 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f3d7ba30038785f88ca0646c63b6c13d, ASSIGN}] 2023-07-24 12:14:56,654 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=hbase:namespace, region=f3d7ba30038785f88ca0646c63b6c13d, ASSIGN 2023-07-24 12:14:56,655 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=5, ppid=4, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=hbase:namespace, region=f3d7ba30038785f88ca0646c63b6c13d, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,34575,1690200894532; forceNewPlan=false, retain=false 2023-07-24 12:14:56,808 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 12:14:56,808 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:56,809 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690200896808"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200896808"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200896808"}]},"ts":"1690200896808"} 2023-07-24 12:14:56,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=6, ppid=5, state=RUNNABLE; OpenRegionProcedure f3d7ba30038785f88ca0646c63b6c13d, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:14:56,969 DEBUG [RSProcedureDispatcher-pool-1] master.ServerManager(712): New admin connection to jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:56,969 DEBUG [RSProcedureDispatcher-pool-1] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 12:14:56,973 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:49032, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 12:14:56,980 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d. 2023-07-24 12:14:56,980 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f3d7ba30038785f88ca0646c63b6c13d, NAME => 'hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d.', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:14:56,981 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:14:56,982 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table namespace f3d7ba30038785f88ca0646c63b6c13d 2023-07-24 12:14:56,982 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:14:56,982 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for f3d7ba30038785f88ca0646c63b6c13d 2023-07-24 12:14:56,982 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for f3d7ba30038785f88ca0646c63b6c13d 2023-07-24 12:14:56,984 INFO [StoreOpener-f3d7ba30038785f88ca0646c63b6c13d-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family info of region f3d7ba30038785f88ca0646c63b6c13d 2023-07-24 12:14:56,987 DEBUG [StoreOpener-f3d7ba30038785f88ca0646c63b6c13d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/namespace/f3d7ba30038785f88ca0646c63b6c13d/info 2023-07-24 12:14:56,987 DEBUG [StoreOpener-f3d7ba30038785f88ca0646c63b6c13d-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/namespace/f3d7ba30038785f88ca0646c63b6c13d/info 2023-07-24 12:14:56,987 INFO [StoreOpener-f3d7ba30038785f88ca0646c63b6c13d-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f3d7ba30038785f88ca0646c63b6c13d columnFamilyName info 2023-07-24 12:14:56,988 INFO [StoreOpener-f3d7ba30038785f88ca0646c63b6c13d-1] regionserver.HStore(310): Store=f3d7ba30038785f88ca0646c63b6c13d/info, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:14:56,990 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/namespace/f3d7ba30038785f88ca0646c63b6c13d 2023-07-24 12:14:56,991 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/namespace/f3d7ba30038785f88ca0646c63b6c13d 2023-07-24 12:14:56,996 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for f3d7ba30038785f88ca0646c63b6c13d 2023-07-24 12:14:57,000 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/namespace/f3d7ba30038785f88ca0646c63b6c13d/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 12:14:57,001 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened f3d7ba30038785f88ca0646c63b6c13d; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60199923, jitterRate=-0.10295124351978302}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 12:14:57,001 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for f3d7ba30038785f88ca0646c63b6c13d: 2023-07-24 12:14:57,003 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d., pid=6, masterSystemTime=1690200896968 2023-07-24 12:14:57,007 DEBUG [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d. 2023-07-24 12:14:57,008 INFO [RS_OPEN_PRIORITY_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d. 2023-07-24 12:14:57,009 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=5 updating hbase:meta row=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:57,009 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d.","families":{"info":[{"qualifier":"regioninfo","vlen":41,"tag":[],"timestamp":"1690200897009"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200897009"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200897009"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200897009"}]},"ts":"1690200897009"} 2023-07-24 12:14:57,016 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=6, resume processing ppid=5 2023-07-24 12:14:57,017 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=6, ppid=5, state=SUCCESS; OpenRegionProcedure f3d7ba30038785f88ca0646c63b6c13d, server=jenkins-hbase6.apache.org,34575,1690200894532 in 199 msec 2023-07-24 12:14:57,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=5, resume processing ppid=4 2023-07-24 12:14:57,021 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=5, ppid=4, state=SUCCESS; TransitRegionStateProcedure table=hbase:namespace, region=f3d7ba30038785f88ca0646c63b6c13d, ASSIGN in 367 msec 2023-07-24 12:14:57,022 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 12:14:57,022 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"hbase:namespace","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200897022"}]},"ts":"1690200897022"} 2023-07-24 12:14:57,024 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=hbase:namespace, state=ENABLED in hbase:meta 2023-07-24 12:14:57,029 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=4, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=hbase:namespace execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 12:14:57,031 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace in 556 msec 2023-07-24 12:14:57,086 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/namespace 2023-07-24 12:14:57,088 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/namespace 2023-07-24 12:14:57,088 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 12:14:57,109 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 12:14:57,112 INFO [RS-EventLoopGroup-4-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:49034, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 12:14:57,124 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=7, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=default 2023-07-24 12:14:57,139 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-24 12:14:57,145 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default in 28 msec 2023-07-24 12:14:57,157 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] procedure2.ProcedureExecutor(1029): Stored pid=8, state=RUNNABLE:CREATE_NAMESPACE_PREPARE; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:14:57,169 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/namespace 2023-07-24 12:14:57,174 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase in 16 msec 2023-07-24 12:14:57,183 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/default 2023-07-24 12:14:57,185 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/namespace/hbase 2023-07-24 12:14:57,186 INFO [master/jenkins-hbase6:0:becomeActiveMaster] master.HMaster(1083): Master has completed initialization 2.557sec 2023-07-24 12:14:57,189 INFO [master/jenkins-hbase6:0:becomeActiveMaster] quotas.MasterQuotaManager(97): Quota support disabled 2023-07-24 12:14:57,190 INFO [master/jenkins-hbase6:0:becomeActiveMaster] slowlog.SlowLogMasterService(57): Slow/Large requests logging to system table hbase:slowlog is disabled. Quitting. 2023-07-24 12:14:57,190 INFO [master/jenkins-hbase6:0:becomeActiveMaster] zookeeper.ZKWatcher(269): not a secure deployment, proceeding 2023-07-24 12:14:57,191 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase6.apache.org,44983,1690200893288-ExpiredMobFileCleanerChore, period=86400, unit=SECONDS is enabled. 2023-07-24 12:14:57,192 INFO [master/jenkins-hbase6:0:becomeActiveMaster] hbase.ChoreService(166): Chore ScheduledChore name=jenkins-hbase6.apache.org,44983,1690200893288-MobCompactionChore, period=604800, unit=SECONDS is enabled. 2023-07-24 12:14:57,201 DEBUG [Listener at localhost/37279] zookeeper.ReadOnlyZKClient(139): Connect 0x50c18f6e to 127.0.0.1:58848 with session timeout=90000ms, retries 30, retry interval 1000ms, keepAlive=60000ms 2023-07-24 12:14:57,201 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster] master.HMaster(1175): Balancer post startup initialization complete, took 0 seconds 2023-07-24 12:14:57,205 DEBUG [Listener at localhost/37279] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@56fdfa18, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=null 2023-07-24 12:14:57,219 DEBUG [hconnection-0x3d545144-shared-pool-0] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 12:14:57,234 INFO [RS-EventLoopGroup-3-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:38668, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 12:14:57,245 INFO [Listener at localhost/37279] hbase.HBaseTestingUtility(1145): Minicluster is up; activeMaster=jenkins-hbase6.apache.org,44983,1690200893288 2023-07-24 12:14:57,274 INFO [Listener at localhost/37279] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=364, OpenFileDescriptor=703, MaxFileDescriptor=60000, SystemLoadAverage=315, ProcessCount=169, AvailableMemoryMB=1760 2023-07-24 12:14:57,278 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:14:57,286 DEBUG [Listener at localhost/37279] ipc.RpcConnection(124): Using SIMPLE authentication for service=MasterService, sasl=false 2023-07-24 12:14:57,289 INFO [RS-EventLoopGroup-1-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:46316, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=MasterService 2023-07-24 12:14:57,302 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/balancer 2023-07-24 12:14:57,303 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 12:14:57,304 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(492): Client=jenkins//172.31.2.20 set balanceSwitch=false 2023-07-24 12:14:57,309 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,311 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 12:14:57,313 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 12:14:57,316 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:57,317 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766 empty. 2023-07-24 12:14:57,321 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:57,321 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-07-24 12:14:57,344 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/.tabledesc/.tableinfo.0000000001 2023-07-24 12:14:57,345 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => f5e0e5aefac66bded47d55c23d50c766, NAME => 'testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionUnevenDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp 2023-07-24 12:14:57,367 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:14:57,367 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1604): Closing f5e0e5aefac66bded47d55c23d50c766, disabling compactions & flushes 2023-07-24 12:14:57,367 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:57,367 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:57,367 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. after waiting 0 ms 2023-07-24 12:14:57,367 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:57,367 INFO [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:57,367 DEBUG [RegionOpenAndInit-testSplitTableRegionUnevenDaughter-pool-0] regionserver.HRegion(1558): Region close journal for f5e0e5aefac66bded47d55c23d50c766: 2023-07-24 12:14:57,371 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 12:14:57,373 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1690200897373"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200897373"}]},"ts":"1690200897373"} 2023-07-24 12:14:57,376 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 12:14:57,377 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 12:14:57,378 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200897378"}]},"ts":"1690200897378"} 2023-07-24 12:14:57,380 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLING in hbase:meta 2023-07-24 12:14:57,384 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase6.apache.org=0} racks are {/default-rack=0} 2023-07-24 12:14:57,385 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 12:14:57,385 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 12:14:57,385 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 12:14:57,385 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 12:14:57,385 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f5e0e5aefac66bded47d55c23d50c766, ASSIGN}] 2023-07-24 12:14:57,388 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f5e0e5aefac66bded47d55c23d50c766, ASSIGN 2023-07-24 12:14:57,389 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=10, ppid=9, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f5e0e5aefac66bded47d55c23d50c766, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,36983,1690200894572; forceNewPlan=false, retain=false 2023-07-24 12:14:57,540 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 12:14:57,541 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=f5e0e5aefac66bded47d55c23d50c766, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:57,541 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1690200897541"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200897541"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200897541"}]},"ts":"1690200897541"} 2023-07-24 12:14:57,544 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=11, ppid=10, state=RUNNABLE; OpenRegionProcedure f5e0e5aefac66bded47d55c23d50c766, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:14:57,698 DEBUG [RSProcedureDispatcher-pool-2] master.ServerManager(712): New admin connection to jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:57,698 DEBUG [RSProcedureDispatcher-pool-2] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 12:14:57,700 INFO [RS-EventLoopGroup-5-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:48782, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 12:14:57,706 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:57,706 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f5e0e5aefac66bded47d55c23d50c766, NAME => 'testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766.', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:14:57,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:14:57,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:57,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:14:57,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:57,707 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:57,709 INFO [StoreOpener-f5e0e5aefac66bded47d55c23d50c766-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:57,711 DEBUG [StoreOpener-f5e0e5aefac66bded47d55c23d50c766-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1 2023-07-24 12:14:57,712 DEBUG [StoreOpener-f5e0e5aefac66bded47d55c23d50c766-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1 2023-07-24 12:14:57,712 INFO [StoreOpener-f5e0e5aefac66bded47d55c23d50c766-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f5e0e5aefac66bded47d55c23d50c766 columnFamilyName cf1 2023-07-24 12:14:57,713 INFO [StoreOpener-f5e0e5aefac66bded47d55c23d50c766-1] regionserver.HStore(310): Store=f5e0e5aefac66bded47d55c23d50c766/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:14:57,713 INFO [StoreOpener-f5e0e5aefac66bded47d55c23d50c766-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:57,715 DEBUG [StoreOpener-f5e0e5aefac66bded47d55c23d50c766-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2 2023-07-24 12:14:57,715 DEBUG [StoreOpener-f5e0e5aefac66bded47d55c23d50c766-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2 2023-07-24 12:14:57,716 INFO [StoreOpener-f5e0e5aefac66bded47d55c23d50c766-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f5e0e5aefac66bded47d55c23d50c766 columnFamilyName cf2 2023-07-24 12:14:57,716 INFO [StoreOpener-f5e0e5aefac66bded47d55c23d50c766-1] regionserver.HStore(310): Store=f5e0e5aefac66bded47d55c23d50c766/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:14:57,718 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:57,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:57,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:14:57,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:57,728 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 12:14:57,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened f5e0e5aefac66bded47d55c23d50c766; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65120342, jitterRate=-0.02963128685951233}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:14:57,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for f5e0e5aefac66bded47d55c23d50c766: 2023-07-24 12:14:57,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766., pid=11, masterSystemTime=1690200897697 2023-07-24 12:14:57,737 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:57,738 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:57,738 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=10 updating hbase:meta row=f5e0e5aefac66bded47d55c23d50c766, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:57,739 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1690200897738"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200897738"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200897738"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200897738"}]},"ts":"1690200897738"} 2023-07-24 12:14:57,745 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=11, resume processing ppid=10 2023-07-24 12:14:57,745 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=11, ppid=10, state=SUCCESS; OpenRegionProcedure f5e0e5aefac66bded47d55c23d50c766, server=jenkins-hbase6.apache.org,36983,1690200894572 in 197 msec 2023-07-24 12:14:57,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=10, resume processing ppid=9 2023-07-24 12:14:57,748 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=10, ppid=9, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f5e0e5aefac66bded47d55c23d50c766, ASSIGN in 360 msec 2023-07-24 12:14:57,749 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 12:14:57,750 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200897749"}]},"ts":"1690200897749"} 2023-07-24 12:14:57,752 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=ENABLED in hbase:meta 2023-07-24 12:14:57,754 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=9, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionUnevenDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 12:14:57,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter in 447 msec 2023-07-24 12:14:57,814 DEBUG [Listener at localhost/37279] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 12:14:57,819 INFO [RS-EventLoopGroup-5-3] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:48798, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 12:14:57,839 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:14:57,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,846 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-24 12:14:57,846 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:57,846 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-24 12:14:57,846 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-24 12:14:57,847 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,847 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,850 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:57,850 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:57,850 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:57,850 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:57,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:57,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:57,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:57,850 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:57,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:57,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:57,850 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:57,850 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:14:57,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,851 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,851 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,851 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:57,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:57,852 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:57,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,852 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,852 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,854 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,854 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,854 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,854 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:57,854 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:57,854 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:57,855 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:57,855 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:57,854 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:57,855 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:57,855 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:57,855 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:57,855 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:57,855 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:57,855 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:57,856 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:57,856 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:14:57,856 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:57,856 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:14:57,856 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. started... 2023-07-24 12:14:57,858 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f5e0e5aefac66bded47d55c23d50c766 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 12:14:57,858 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,858 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,858 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,858 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:57,858 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:57,858 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:57,930 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1 2023-07-24 12:14:57,989 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf2/692579edb5264888aa4103b744a2f08b 2023-07-24 12:14:58,001 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1 2023-07-24 12:14:58,012 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:14:58,015 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf2/692579edb5264888aa4103b744a2f08b as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/692579edb5264888aa4103b744a2f08b 2023-07-24 12:14:58,024 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/692579edb5264888aa4103b744a2f08b, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:14:58,026 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for f5e0e5aefac66bded47d55c23d50c766 in 168ms, sequenceid=6, compaction requested=false 2023-07-24 12:14:58,028 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionUnevenDaughter' 2023-07-24 12:14:58,030 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f5e0e5aefac66bded47d55c23d50c766: 2023-07-24 12:14:58,030 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:58,030 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:14:58,031 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:14:58,031 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,031 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:58,031 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:58,033 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,033 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:58,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:58,034 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,034 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:58,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:58,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:58,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:58,038 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-24 12:14:58,038 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-24 12:14:58,038 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@45ab348a[Count = 0] remaining members to acquire global barrier 2023-07-24 12:14:58,038 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,046 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,046 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,046 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,046 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:58,047 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,047 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:58,047 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:58,047 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,047 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:58,047 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:14:58,047 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:58,047 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:58,047 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:14:58,047 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:14:58,047 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:14:58,052 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:58,052 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,052 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,052 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:58,052 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:58,052 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:58,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:58,053 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:58,053 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:58,053 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,053 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:58,054 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:58,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,055 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,056 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:58,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,057 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,058 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:14:58,058 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:14:58,058 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-24 12:14:58,058 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:14:58,058 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:14:58,058 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,058 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:14:58,061 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,061 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:58,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:58,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,061 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,061 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,061 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,061 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,061 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:58,061 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,061 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,061 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,062 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,062 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:58,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,062 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,063 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,063 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,064 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:58,065 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,065 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,065 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,066 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,066 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,076 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,076 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,076 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,076 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,076 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,076 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,076 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,076 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,076 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,076 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,076 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,076 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,076 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,076 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-24 12:14:58,077 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:14:58,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,076 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,076 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,079 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,079 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,079 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,080 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,079 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:14:58,080 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,080 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,080 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,080 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:14:58,181 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:14:58,185 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:14:58,211 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:14:58,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:14:58,214 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,214 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-24 12:14:58,214 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,215 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-24 12:14:58,215 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-24 12:14:58,216 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,216 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,218 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,218 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,218 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,218 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,218 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,218 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,218 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:14:58,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,219 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,219 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,219 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:58,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:58,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:58,220 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,220 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,220 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:58,220 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,220 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,220 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:58,220 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:58,220 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:58,220 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,221 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:58,221 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:58,221 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:58,221 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,221 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,222 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:58,222 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:14:58,222 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:58,222 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:58,222 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:58,222 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:58,222 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. started... 2023-07-24 12:14:58,222 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:14:58,223 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:58,223 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f5e0e5aefac66bded47d55c23d50c766 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:14:58,223 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:58,226 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,227 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,227 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:58,227 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,227 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,227 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:58,249 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf1/ed220ebfec37496bb6f5efd49e84532e 2023-07-24 12:14:58,283 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf2/9a102cd162f249a087b7a5984ccba105 2023-07-24 12:14:58,292 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf1/ed220ebfec37496bb6f5efd49e84532e as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ed220ebfec37496bb6f5efd49e84532e 2023-07-24 12:14:58,301 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ed220ebfec37496bb6f5efd49e84532e, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:14:58,303 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf2/9a102cd162f249a087b7a5984ccba105 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/9a102cd162f249a087b7a5984ccba105 2023-07-24 12:14:58,316 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/9a102cd162f249a087b7a5984ccba105, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:14:58,317 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f5e0e5aefac66bded47d55c23d50c766 in 94ms, sequenceid=19, compaction requested=false 2023-07-24 12:14:58,317 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f5e0e5aefac66bded47d55c23d50c766: 2023-07-24 12:14:58,317 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:58,318 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:14:58,318 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:14:58,318 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,318 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:58,318 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:58,320 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,320 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:58,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:58,321 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,321 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:58,321 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:58,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:58,322 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,323 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,325 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:58,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-24 12:14:58,326 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@76965f07[Count = 0] remaining members to acquire global barrier 2023-07-24 12:14:58,326 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-24 12:14:58,326 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,329 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,329 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,329 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,329 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,330 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,330 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:58,330 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:58,330 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:58,330 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:58,330 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:58,330 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:14:58,330 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:14:58,330 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:14:58,330 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:58,330 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:14:58,333 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:58,333 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,333 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,333 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:58,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:58,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:58,333 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:58,333 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:58,334 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:58,334 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,334 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:58,334 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:58,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:58,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:58,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:14:58,340 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:14:58,341 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-24 12:14:58,341 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:14:58,341 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:14:58,341 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,341 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:14:58,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:58,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:58,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:58,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,345 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,346 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:58,346 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,348 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:58,348 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,349 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,349 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,350 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,350 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,359 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,359 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,359 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,359 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,359 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,359 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,359 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,360 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,359 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,359 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,360 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,360 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-24 12:14:58,360 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:14:58,360 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,360 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,360 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,361 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,361 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:14:58,361 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:14:58,361 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,361 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,362 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,362 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,362 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,362 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,362 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,362 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,362 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,462 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:14:58,463 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:14:58,486 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:14:58,488 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:14:58,490 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,490 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-24 12:14:58,491 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,491 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-24 12:14:58,491 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-24 12:14:58,492 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,492 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,493 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,493 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,493 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,493 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,493 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,494 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,494 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,494 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:14:58,493 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,494 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,495 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,495 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,495 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,495 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:58,495 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,495 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,496 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,496 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:58,496 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:58,496 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:58,496 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,496 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,496 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:58,496 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:58,496 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,496 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,497 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:58,497 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:58,497 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,496 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,497 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:14:58,497 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:58,497 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:58,497 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:14:58,497 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:58,497 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:58,497 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. started... 2023-07-24 12:14:58,497 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:58,498 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:58,498 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:58,498 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f5e0e5aefac66bded47d55c23d50c766 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:14:58,500 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,500 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,500 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,500 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:58,500 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,500 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:58,521 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf1/a115aba766a9442a8956f79a6f674f2f 2023-07-24 12:14:58,547 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf2/c42ea765e2b24588a4a836cf8f33d6b0 2023-07-24 12:14:58,556 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf1/a115aba766a9442a8956f79a6f674f2f as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/a115aba766a9442a8956f79a6f674f2f 2023-07-24 12:14:58,563 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/a115aba766a9442a8956f79a6f674f2f, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:14:58,565 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf2/c42ea765e2b24588a4a836cf8f33d6b0 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/c42ea765e2b24588a4a836cf8f33d6b0 2023-07-24 12:14:58,572 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/c42ea765e2b24588a4a836cf8f33d6b0, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:14:58,574 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f5e0e5aefac66bded47d55c23d50c766 in 76ms, sequenceid=32, compaction requested=true 2023-07-24 12:14:58,574 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f5e0e5aefac66bded47d55c23d50c766: 2023-07-24 12:14:58,574 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:58,574 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:14:58,575 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:14:58,575 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,575 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:58,575 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:58,576 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,576 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,576 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:58,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:58,577 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,577 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:58,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:58,577 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:58,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:58,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-24 12:14:58,579 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@464e1ab4[Count = 0] remaining members to acquire global barrier 2023-07-24 12:14:58,579 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-24 12:14:58,579 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,582 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,582 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,582 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,582 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:58,582 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,582 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:58,582 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:58,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,582 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:58,582 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:14:58,582 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:58,582 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:58,582 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:14:58,582 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:14:58,582 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,582 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:14:58,585 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:58,585 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,585 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:58,586 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,586 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:58,586 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:58,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:58,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:58,586 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,586 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,586 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:58,588 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:58,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:58,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:58,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,589 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:58,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:14:58,592 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:14:58,592 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-24 12:14:58,592 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:14:58,592 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:14:58,592 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,592 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:14:58,594 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,594 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,594 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,594 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,594 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,594 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,594 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,594 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:58,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:58,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,595 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,595 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,595 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:58,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,596 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,596 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:58,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,599 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:58,599 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,599 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,600 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,600 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,601 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,604 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,604 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,604 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,604 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,604 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,604 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-24 12:14:58,605 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:14:58,604 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,605 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,605 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,605 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:14:58,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,604 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,605 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:14:58,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,605 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,606 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,606 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,606 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,606 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,606 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,606 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,606 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,706 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:14:58,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:14:58,731 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:14:58,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:14:58,734 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,734 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-24 12:14:58,734 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,735 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-24 12:14:58,735 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-24 12:14:58,736 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,736 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,737 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,737 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,738 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,738 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,738 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,738 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,738 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:14:58,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,739 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,739 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,739 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:58,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:58,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:58,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,740 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,740 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:58,740 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,740 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:58,740 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,744 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:58,744 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:58,744 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:58,744 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:58,744 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:58,744 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,744 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,744 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:58,744 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:58,744 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,744 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,745 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:58,746 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:58,746 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:58,746 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:58,746 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:14:58,746 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:14:58,746 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:58,746 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. started... 2023-07-24 12:14:58,746 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f5e0e5aefac66bded47d55c23d50c766 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:14:58,747 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,749 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,749 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,749 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:58,749 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,749 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:58,769 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf1/df90e6c0608840eba909e7abe5f3a50a 2023-07-24 12:14:58,800 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf2/a406428afd6848e1ac9573741b2f2872 2023-07-24 12:14:58,809 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf1/df90e6c0608840eba909e7abe5f3a50a as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/df90e6c0608840eba909e7abe5f3a50a 2023-07-24 12:14:58,817 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/df90e6c0608840eba909e7abe5f3a50a, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:14:58,818 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf2/a406428afd6848e1ac9573741b2f2872 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a406428afd6848e1ac9573741b2f2872 2023-07-24 12:14:58,826 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a406428afd6848e1ac9573741b2f2872, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:14:58,827 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f5e0e5aefac66bded47d55c23d50c766 in 81ms, sequenceid=45, compaction requested=true 2023-07-24 12:14:58,827 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f5e0e5aefac66bded47d55c23d50c766: 2023-07-24 12:14:58,827 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:58,827 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:14:58,828 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:14:58,828 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,828 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:58,828 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:58,835 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,835 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:58,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:58,835 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,835 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:58,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:58,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:58,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:58,837 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-24 12:14:58,837 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@61575cda[Count = 0] remaining members to acquire global barrier 2023-07-24 12:14:58,837 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-24 12:14:58,837 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,839 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,839 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,839 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,839 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,839 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:58,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,839 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,839 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:58,839 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:58,839 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:14:58,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,839 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:14:58,840 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:58,840 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:58,839 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:58,840 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:14:58,840 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:14:58,841 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:58,841 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,841 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:58,844 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:58,844 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,845 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:58,845 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:58,845 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,845 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,845 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:58,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:58,845 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:58,846 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,847 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,848 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:58,849 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,850 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:14:58,851 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:14:58,851 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-24 12:14:58,851 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:14:58,851 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:14:58,851 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,851 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:14:58,853 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,853 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,853 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,853 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,853 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,853 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,853 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:58,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:58,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,853 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,854 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:58,854 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,855 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,855 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,855 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,857 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:58,858 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,858 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,862 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,862 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,862 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,862 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,862 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,862 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,862 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,862 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,862 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,862 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,862 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:58,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,862 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-24 12:14:58,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,862 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:14:58,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:58,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:58,863 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:58,864 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:14:58,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:58,864 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:14:58,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,865 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,964 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:14:58,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:14:58,988 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:14:58,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:14:58,990 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,990 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-24 12:14:58,990 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,991 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-24 12:14:58,991 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-24 12:14:58,992 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,992 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,994 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,994 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,994 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:58,994 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,994 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,994 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:58,995 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:58,995 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:14:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,995 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,996 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,996 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,996 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:58,996 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,996 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,997 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,997 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,997 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:58,997 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:58,997 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,997 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:58,997 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:58,997 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:58,997 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,997 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:58,999 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:58,999 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:59,000 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:59,000 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:59,000 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:14:59,000 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:14:59,000 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:59,000 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:59,000 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,000 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. started... 2023-07-24 12:14:59,000 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:59,000 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:59,002 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:59,002 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:59,002 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f5e0e5aefac66bded47d55c23d50c766 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:14:59,002 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:59,003 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,004 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,004 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,005 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:59,005 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,005 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:59,023 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf1/d41a8a9025e246d89f1527a5f8d3faf0 2023-07-24 12:14:59,050 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf2/a80050507df24ed78b174bd165a5a930 2023-07-24 12:14:59,058 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf1/d41a8a9025e246d89f1527a5f8d3faf0 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/d41a8a9025e246d89f1527a5f8d3faf0 2023-07-24 12:14:59,066 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/d41a8a9025e246d89f1527a5f8d3faf0, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:14:59,067 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf2/a80050507df24ed78b174bd165a5a930 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a80050507df24ed78b174bd165a5a930 2023-07-24 12:14:59,075 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a80050507df24ed78b174bd165a5a930, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:14:59,077 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f5e0e5aefac66bded47d55c23d50c766 in 77ms, sequenceid=58, compaction requested=true 2023-07-24 12:14:59,077 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f5e0e5aefac66bded47d55c23d50c766: 2023-07-24 12:14:59,077 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:59,077 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:14:59,078 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:14:59,078 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,078 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:59,078 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:59,080 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,080 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:59,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:59,080 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,080 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:59,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:59,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:59,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:59,082 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-24 12:14:59,082 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7dc42b1e[Count = 0] remaining members to acquire global barrier 2023-07-24 12:14:59,082 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-24 12:14:59,083 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,085 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,085 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,085 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,085 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,085 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:14:59,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,085 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:59,085 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:59,085 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:59,085 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,085 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:14:59,085 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:59,085 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,085 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:14:59,085 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:59,086 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:59,086 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:14:59,087 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:59,087 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:59,087 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:59,088 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,088 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:59,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:59,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:59,088 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:59,088 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:59,088 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:59,089 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:59,089 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:59,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:59,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:59,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:59,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:14:59,093 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:14:59,093 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-24 12:14:59,093 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:14:59,093 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:14:59,093 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,093 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:14:59,095 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,095 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,095 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,095 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:59,095 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,095 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:59,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:59,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:59,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,095 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,095 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,095 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:59,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:59,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:59,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:59,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:59,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:59,096 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:59,096 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:59,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,097 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:59,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,099 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:59,099 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,100 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,107 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,107 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:59,107 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,107 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:59,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:59,107 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,107 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:59,107 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:59,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:59,107 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:59,107 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,107 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:59,108 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:59,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-24 12:14:59,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:14:59,108 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:59,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:59,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:59,108 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:59,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:59,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:59,108 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,109 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,109 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:14:59,109 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:59,109 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:59,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:59,109 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:14:59,109 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:59,110 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,210 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:14:59,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:14:59,234 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:14:59,236 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionUnevenDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:14:59,237 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,237 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionUnevenDaughter' 2023-07-24 12:14:59,237 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:59,238 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' 2023-07-24 12:14:59,238 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionUnevenDaughter', kicking off acquire phase on members. 2023-07-24 12:14:59,238 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,238 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,240 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:59,240 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:59,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:59,240 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:59,240 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:59,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:59,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:59,240 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,240 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:14:59,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:59,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:59,241 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:59,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,241 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,241 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,241 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,241 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:59,241 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,242 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:59,242 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:59,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:59,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:14:59,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,242 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:59,242 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,242 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,242 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:59,243 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,243 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:59,243 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:59,243 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,243 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:59,243 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionUnevenDaughter is still present, removing 2023-07-24 12:14:59,243 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:59,243 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,244 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:59,244 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionUnevenDaughter' with timeout 60000ms 2023-07-24 12:14:59,244 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:14:59,244 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:59,244 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:59,244 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionUnevenDaughter' starting 'acquire' stage 2023-07-24 12:14:59,245 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:14:59,245 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:14:59,245 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:59,245 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. started... 2023-07-24 12:14:59,245 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing f5e0e5aefac66bded47d55c23d50c766 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:14:59,246 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,248 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,248 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,248 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:59,248 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,248 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:59,265 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf1/3a5e9c1e694d4cf8a1e9226147da4628 2023-07-24 12:14:59,285 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf2/bbb0688273a74cdf80bec7200cbc5879 2023-07-24 12:14:59,293 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf1/3a5e9c1e694d4cf8a1e9226147da4628 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/3a5e9c1e694d4cf8a1e9226147da4628 2023-07-24 12:14:59,301 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/3a5e9c1e694d4cf8a1e9226147da4628, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:14:59,302 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf2/bbb0688273a74cdf80bec7200cbc5879 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/bbb0688273a74cdf80bec7200cbc5879 2023-07-24 12:14:59,310 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/bbb0688273a74cdf80bec7200cbc5879, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:14:59,311 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for f5e0e5aefac66bded47d55c23d50c766 in 66ms, sequenceid=71, compaction requested=true 2023-07-24 12:14:59,312 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for f5e0e5aefac66bded47d55c23d50c766: 2023-07-24 12:14:59,312 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:59,312 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:14:59,314 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:14:59,314 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,314 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionUnevenDaughter' locally acquired 2023-07-24 12:14:59,314 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionUnevenDaughter) in zk 2023-07-24 12:14:59,315 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,315 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:59,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:59,316 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,316 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionUnevenDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:14:59,316 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:59,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:59,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,317 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:59,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitTableRegionUnevenDaughter' on coordinator 2023-07-24 12:14:59,318 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@396e2dc6[Count = 0] remaining members to acquire global barrier 2023-07-24 12:14:59,318 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionUnevenDaughter' starting 'in-barrier' execution. 2023-07-24 12:14:59,319 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,320 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,320 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,320 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,320 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:59,320 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,320 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,320 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:14:59,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,320 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,320 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:59,320 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionUnevenDaughter' received 'reached' from coordinator. 2023-07-24 12:14:59,320 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:59,320 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:59,320 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:14:59,320 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:14:59,320 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionUnevenDaughter' locally completed 2023-07-24 12:14:59,321 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionUnevenDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:14:59,322 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:59,322 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:59,322 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:59,326 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:59,326 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionUnevenDaughter' has notified controller of completion 2023-07-24 12:14:59,326 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,326 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:59,326 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:59,326 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:59,327 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:59,326 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:59,327 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionUnevenDaughter' completed. 2023-07-24 12:14:59,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:59,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:59,328 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,329 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:59,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,330 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,331 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionUnevenDaughter' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:14:59,332 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitTableRegionUnevenDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:14:59,332 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionUnevenDaughter' execution completed 2023-07-24 12:14:59,332 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:14:59,332 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:14:59,332 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,332 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionUnevenDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:14:59,334 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,334 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:14:59,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:14:59,334 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,334 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:59,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,334 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,334 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:14:59,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,335 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,334 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:59,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:59,334 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,334 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:59,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:59,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:59,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:59,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,335 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:59,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:59,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:14:59,336 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,336 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,337 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:14:59,337 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,338 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,338 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,338 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,341 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,341 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,341 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:59,341 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,341 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:59,341 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:59,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:59,341 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,341 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:59,342 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,341 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:14:59,342 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionUnevenDaughter' 2023-07-24 12:14:59,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:59,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:59,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:59,342 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:59,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:59,342 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:14:59,342 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,342 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:14:59,342 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:14:59,342 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:14:59,342 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:14:59,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:59,343 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionUnevenDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:14:59,342 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:14:59,343 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:14:59,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:59,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:14:59,344 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,344 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionUnevenDaughter 2023-07-24 12:14:59,345 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:14:59,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:14:59,444 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:14:59,445 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:14:59,469 DEBUG [Listener at localhost/37279] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,478 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:14:59,479 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:14:59,479 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:14:59,479 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=12, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:14:59,486 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f5e0e5aefac66bded47d55c23d50c766, UNASSIGN}] 2023-07-24 12:14:59,488 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=13, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f5e0e5aefac66bded47d55c23d50c766, UNASSIGN 2023-07-24 12:14:59,489 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=f5e0e5aefac66bded47d55c23d50c766, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:14:59,489 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1690200899489"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200899489"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200899489"}]},"ts":"1690200899489"} 2023-07-24 12:14:59,491 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=14, ppid=13, state=RUNNABLE; CloseRegionProcedure f5e0e5aefac66bded47d55c23d50c766, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:14:59,652 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing f5e0e5aefac66bded47d55c23d50c766, disabling compactions & flushes 2023-07-24 12:14:59,652 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:59,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:59,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. after waiting 0 ms 2023-07-24 12:14:59,652 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:59,652 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2745): Flushing f5e0e5aefac66bded47d55c23d50c766 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-24 12:14:59,673 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf1/ea30a72aa0544e16a31b0a0d8774946e 2023-07-24 12:14:59,697 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf2/6bb994da2b534b8c959238f2707401dc 2023-07-24 12:14:59,705 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf1/ea30a72aa0544e16a31b0a0d8774946e as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ea30a72aa0544e16a31b0a0d8774946e 2023-07-24 12:14:59,712 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ea30a72aa0544e16a31b0a0d8774946e, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 12:14:59,714 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.tmp/cf2/6bb994da2b534b8c959238f2707401dc as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/6bb994da2b534b8c959238f2707401dc 2023-07-24 12:14:59,722 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/6bb994da2b534b8c959238f2707401dc, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 12:14:59,723 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for f5e0e5aefac66bded47d55c23d50c766 in 71ms, sequenceid=82, compaction requested=true 2023-07-24 12:14:59,746 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-24 12:14:59,747 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:14:59,748 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. 2023-07-24 12:14:59,748 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for f5e0e5aefac66bded47d55c23d50c766: 2023-07-24 12:14:59,751 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,752 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=13 updating hbase:meta row=f5e0e5aefac66bded47d55c23d50c766, regionState=CLOSED 2023-07-24 12:14:59,752 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1690200899752"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200899752"}]},"ts":"1690200899752"} 2023-07-24 12:14:59,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=14, resume processing ppid=13 2023-07-24 12:14:59,757 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=14, ppid=13, state=SUCCESS; CloseRegionProcedure f5e0e5aefac66bded47d55c23d50c766, server=jenkins-hbase6.apache.org,36983,1690200894572 in 264 msec 2023-07-24 12:14:59,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=13, resume processing ppid=12 2023-07-24 12:14:59,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=13, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f5e0e5aefac66bded47d55c23d50c766, UNASSIGN in 271 msec 2023-07-24 12:14:59,781 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=12 splitting 14 storefiles, region=f5e0e5aefac66bded47d55c23d50c766, threads=14 2023-07-24 12:14:59,782 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/692579edb5264888aa4103b744a2f08b for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,783 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/6bb994da2b534b8c959238f2707401dc for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,783 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/9a102cd162f249a087b7a5984ccba105 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,784 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a406428afd6848e1ac9573741b2f2872 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,784 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a80050507df24ed78b174bd165a5a930 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,784 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/bbb0688273a74cdf80bec7200cbc5879 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,784 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/c42ea765e2b24588a4a836cf8f33d6b0 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,784 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/3a5e9c1e694d4cf8a1e9226147da4628 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,785 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/a115aba766a9442a8956f79a6f674f2f for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,785 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,787 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/d41a8a9025e246d89f1527a5f8d3faf0 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,790 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/df90e6c0608840eba909e7abe5f3a50a for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,791 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ea30a72aa0544e16a31b0a0d8774946e for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,795 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=12 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ed220ebfec37496bb6f5efd49e84532e for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,816 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/bbb0688273a74cdf80bec7200cbc5879, top=true 2023-07-24 12:14:59,830 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a80050507df24ed78b174bd165a5a930, top=true 2023-07-24 12:14:59,840 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ea30a72aa0544e16a31b0a0d8774946e, top=true 2023-07-24 12:14:59,869 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/6bb994da2b534b8c959238f2707401dc, top=true 2023-07-24 12:14:59,872 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/3a5e9c1e694d4cf8a1e9226147da4628, top=true 2023-07-24 12:14:59,874 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/d41a8a9025e246d89f1527a5f8d3faf0, top=true 2023-07-24 12:14:59,876 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.splits/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-bbb0688273a74cdf80bec7200cbc5879 for child: f1b51e1d4f8265994182eae453729fd8, parent: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,876 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/bbb0688273a74cdf80bec7200cbc5879 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,915 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.splits/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-6bb994da2b534b8c959238f2707401dc for child: f1b51e1d4f8265994182eae453729fd8, parent: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,915 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.splits/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-a80050507df24ed78b174bd165a5a930 for child: f1b51e1d4f8265994182eae453729fd8, parent: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,915 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/6bb994da2b534b8c959238f2707401dc for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,915 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a80050507df24ed78b174bd165a5a930 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,923 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.splits/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-ea30a72aa0544e16a31b0a0d8774946e for child: f1b51e1d4f8265994182eae453729fd8, parent: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,923 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ea30a72aa0544e16a31b0a0d8774946e for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,923 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.splits/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-d41a8a9025e246d89f1527a5f8d3faf0 for child: f1b51e1d4f8265994182eae453729fd8, parent: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,924 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/d41a8a9025e246d89f1527a5f8d3faf0 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,930 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/.splits/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-3a5e9c1e694d4cf8a1e9226147da4628 for child: f1b51e1d4f8265994182eae453729fd8, parent: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:14:59,930 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/3a5e9c1e694d4cf8a1e9226147da4628 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:00,057 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ed220ebfec37496bb6f5efd49e84532e for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:00,059 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/692579edb5264888aa4103b744a2f08b for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:00,071 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:00,080 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/9a102cd162f249a087b7a5984ccba105 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:00,083 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/a115aba766a9442a8956f79a6f674f2f for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:00,083 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a406428afd6848e1ac9573741b2f2872 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:00,084 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/df90e6c0608840eba909e7abe5f3a50a for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:00,085 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=12 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/c42ea765e2b24588a4a836cf8f33d6b0 for region: f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:00,085 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=12 split storefiles for region f5e0e5aefac66bded47d55c23d50c766 Daughter A: 8 storefiles, Daughter B: 14 storefiles. 2023-07-24 12:15:00,137 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 12:15:00,139 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 12:15:00,141 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766.","families":{"info":[{"qualifier":"regioninfo","vlen":68,"tag":[],"timestamp":"1690200900141"},{"qualifier":"splitA","vlen":70,"tag":[],"timestamp":"1690200900141"},{"qualifier":"splitB","vlen":70,"tag":[],"timestamp":"1690200900141"}]},"ts":"1690200900141"} 2023-07-24 12:15:00,141 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690200900141"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200900141"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200900141"}]},"ts":"1690200900141"} 2023-07-24 12:15:00,141 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690200900141"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200900141"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200900141"}]},"ts":"1690200900141"} 2023-07-24 12:15:00,191 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7b2eccad2ce02a648b563be8a6c9f93e, ASSIGN}, {pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f1b51e1d4f8265994182eae453729fd8, ASSIGN}] 2023-07-24 12:15:00,192 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f1b51e1d4f8265994182eae453729fd8, ASSIGN 2023-07-24 12:15:00,193 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=16, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f1b51e1d4f8265994182eae453729fd8, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase6.apache.org,36983,1690200894572; forceNewPlan=false, retain=false 2023-07-24 12:15:00,193 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7b2eccad2ce02a648b563be8a6c9f93e, ASSIGN 2023-07-24 12:15:00,194 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=15, ppid=12, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7b2eccad2ce02a648b563be8a6c9f93e, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase6.apache.org,36983,1690200894572; forceNewPlan=false, retain=false 2023-07-24 12:15:00,343 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-24 12:15:00,345 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=f1b51e1d4f8265994182eae453729fd8, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:00,345 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690200900345"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200900345"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200900345"}]},"ts":"1690200900345"} 2023-07-24 12:15:00,347 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=17, ppid=16, state=RUNNABLE; OpenRegionProcedure f1b51e1d4f8265994182eae453729fd8, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:00,351 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=7b2eccad2ce02a648b563be8a6c9f93e, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:00,351 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690200900351"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200900351"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200900351"}]},"ts":"1690200900351"} 2023-07-24 12:15:00,353 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=18, ppid=15, state=RUNNABLE; OpenRegionProcedure 7b2eccad2ce02a648b563be8a6c9f93e, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:00,504 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e. 2023-07-24 12:15:00,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7b2eccad2ce02a648b563be8a6c9f93e, NAME => 'testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.', STARTKEY => '', ENDKEY => '26'} 2023-07-24 12:15:00,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:00,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter 7b2eccad2ce02a648b563be8a6c9f93e 2023-07-24 12:15:00,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:00,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 7b2eccad2ce02a648b563be8a6c9f93e 2023-07-24 12:15:00,505 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 7b2eccad2ce02a648b563be8a6c9f93e 2023-07-24 12:15:00,507 INFO [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7b2eccad2ce02a648b563be8a6c9f93e 2023-07-24 12:15:00,508 DEBUG [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1 2023-07-24 12:15:00,508 DEBUG [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1 2023-07-24 12:15:00,509 INFO [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7b2eccad2ce02a648b563be8a6c9f93e columnFamilyName cf1 2023-07-24 12:15:00,546 DEBUG [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/a115aba766a9442a8956f79a6f674f2f.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/a115aba766a9442a8956f79a6f674f2f-bottom 2023-07-24 12:15:00,554 DEBUG [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1-bottom 2023-07-24 12:15:00,562 DEBUG [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/df90e6c0608840eba909e7abe5f3a50a.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/df90e6c0608840eba909e7abe5f3a50a-bottom 2023-07-24 12:15:00,570 DEBUG [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/ed220ebfec37496bb6f5efd49e84532e.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ed220ebfec37496bb6f5efd49e84532e-bottom 2023-07-24 12:15:00,570 INFO [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] regionserver.HStore(310): Store=7b2eccad2ce02a648b563be8a6c9f93e/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:00,570 INFO [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7b2eccad2ce02a648b563be8a6c9f93e 2023-07-24 12:15:00,572 DEBUG [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2 2023-07-24 12:15:00,572 DEBUG [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2 2023-07-24 12:15:00,572 INFO [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7b2eccad2ce02a648b563be8a6c9f93e columnFamilyName cf2 2023-07-24 12:15:00,588 DEBUG [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/692579edb5264888aa4103b744a2f08b.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/692579edb5264888aa4103b744a2f08b-bottom 2023-07-24 12:15:00,595 DEBUG [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/9a102cd162f249a087b7a5984ccba105.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/9a102cd162f249a087b7a5984ccba105-bottom 2023-07-24 12:15:00,601 DEBUG [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/a406428afd6848e1ac9573741b2f2872.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a406428afd6848e1ac9573741b2f2872-bottom 2023-07-24 12:15:00,608 DEBUG [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/c42ea765e2b24588a4a836cf8f33d6b0.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/c42ea765e2b24588a4a836cf8f33d6b0-bottom 2023-07-24 12:15:00,608 INFO [StoreOpener-7b2eccad2ce02a648b563be8a6c9f93e-1] regionserver.HStore(310): Store=7b2eccad2ce02a648b563be8a6c9f93e/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:00,609 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e 2023-07-24 12:15:00,611 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e 2023-07-24 12:15:00,613 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:15:00,615 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 7b2eccad2ce02a648b563be8a6c9f93e 2023-07-24 12:15:00,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 7b2eccad2ce02a648b563be8a6c9f93e; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59858922, jitterRate=-0.10803255438804626}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:15:00,616 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 7b2eccad2ce02a648b563be8a6c9f93e: 2023-07-24 12:15:00,617 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e., pid=18, masterSystemTime=1690200900500 2023-07-24 12:15:00,619 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:00,620 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 12:15:00,620 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-07-24 12:15:00,620 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 4 store files, 0 compacting, 4 eligible, 16 blocking 2023-07-24 12:15:00,623 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e. 2023-07-24 12:15:00,623 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e. 2023-07-24 12:15:00,623 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1912): 7b2eccad2ce02a648b563be8a6c9f93e/cf2 is initiating minor compaction (all files) 2023-07-24 12:15:00,623 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e. 2023-07-24 12:15:00,623 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1912): 7b2eccad2ce02a648b563be8a6c9f93e/cf1 is initiating minor compaction (all files) 2023-07-24 12:15:00,623 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 7b2eccad2ce02a648b563be8a6c9f93e/cf2 in testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e. 2023-07-24 12:15:00,623 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 7b2eccad2ce02a648b563be8a6c9f93e/cf1 in testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e. 2023-07-24 12:15:00,623 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=15 updating hbase:meta row=7b2eccad2ce02a648b563be8a6c9f93e, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:00,624 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690200900623"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200900623"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200900623"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200900623"}]},"ts":"1690200900623"} 2023-07-24 12:15:00,624 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/ed220ebfec37496bb6f5efd49e84532e.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ed220ebfec37496bb6f5efd49e84532e-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/a115aba766a9442a8956f79a6f674f2f.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/a115aba766a9442a8956f79a6f674f2f-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/df90e6c0608840eba909e7abe5f3a50a.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/df90e6c0608840eba909e7abe5f3a50a-bottom] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/.tmp, totalSize=19.9 K 2023-07-24 12:15:00,624 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/692579edb5264888aa4103b744a2f08b.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/692579edb5264888aa4103b744a2f08b-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/9a102cd162f249a087b7a5984ccba105.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/9a102cd162f249a087b7a5984ccba105-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/c42ea765e2b24588a4a836cf8f33d6b0.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/c42ea765e2b24588a4a836cf8f33d6b0-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/a406428afd6848e1ac9573741b2f2872.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a406428afd6848e1ac9573741b2f2872-bottom] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/.tmp, totalSize=19.9 K 2023-07-24 12:15:00,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e. 2023-07-24 12:15:00,626 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8. 2023-07-24 12:15:00,626 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f1b51e1d4f8265994182eae453729fd8, NAME => 'testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.', STARTKEY => '26', ENDKEY => ''} 2023-07-24 12:15:00,627 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting c7cdcc2a7d3d41a3afba02ba9edc6ec1.f5e0e5aefac66bded47d55c23d50c766, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690200897820 2023-07-24 12:15:00,628 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 692579edb5264888aa4103b744a2f08b.f5e0e5aefac66bded47d55c23d50c766, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690200897820 2023-07-24 12:15:00,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:00,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionUnevenDaughter f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:00,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:00,629 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting ed220ebfec37496bb6f5efd49e84532e.f5e0e5aefac66bded47d55c23d50c766, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690200898188 2023-07-24 12:15:00,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:00,629 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 9a102cd162f249a087b7a5984ccba105.f5e0e5aefac66bded47d55c23d50c766, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690200898188 2023-07-24 12:15:00,629 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:00,630 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting a115aba766a9442a8956f79a6f674f2f.f5e0e5aefac66bded47d55c23d50c766, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690200898464 2023-07-24 12:15:00,630 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting c42ea765e2b24588a4a836cf8f33d6b0.f5e0e5aefac66bded47d55c23d50c766, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690200898464 2023-07-24 12:15:00,631 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting df90e6c0608840eba909e7abe5f3a50a.f5e0e5aefac66bded47d55c23d50c766, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690200898709 2023-07-24 12:15:00,631 INFO [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:00,631 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting a406428afd6848e1ac9573741b2f2872.f5e0e5aefac66bded47d55c23d50c766, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690200898709 2023-07-24 12:15:00,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=18, resume processing ppid=15 2023-07-24 12:15:00,633 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=18, ppid=15, state=SUCCESS; OpenRegionProcedure 7b2eccad2ce02a648b563be8a6c9f93e, server=jenkins-hbase6.apache.org,36983,1690200894572 in 275 msec 2023-07-24 12:15:00,635 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1 2023-07-24 12:15:00,635 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1 2023-07-24 12:15:00,636 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=15, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7b2eccad2ce02a648b563be8a6c9f93e, ASSIGN in 442 msec 2023-07-24 12:15:00,636 INFO [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1b51e1d4f8265994182eae453729fd8 columnFamilyName cf1 2023-07-24 12:15:00,654 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7b2eccad2ce02a648b563be8a6c9f93e#cf1#compaction#14 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:00,659 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] throttle.PressureAwareThroughputController(145): 7b2eccad2ce02a648b563be8a6c9f93e#cf2#compaction#15 average throughput is 0.43 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:00,660 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/a115aba766a9442a8956f79a6f674f2f.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/a115aba766a9442a8956f79a6f674f2f-top 2023-07-24 12:15:00,671 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1-top 2023-07-24 12:15:00,676 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/df90e6c0608840eba909e7abe5f3a50a.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/df90e6c0608840eba909e7abe5f3a50a-top 2023-07-24 12:15:00,683 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/ed220ebfec37496bb6f5efd49e84532e.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ed220ebfec37496bb6f5efd49e84532e-top 2023-07-24 12:15:00,689 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-3a5e9c1e694d4cf8a1e9226147da4628 2023-07-24 12:15:00,704 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-d41a8a9025e246d89f1527a5f8d3faf0 2023-07-24 12:15:00,710 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/.tmp/cf1/f98742cb66294e5b91193eb16cf37c24 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/f98742cb66294e5b91193eb16cf37c24 2023-07-24 12:15:00,713 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-ea30a72aa0544e16a31b0a0d8774946e 2023-07-24 12:15:00,713 INFO [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(310): Store=f1b51e1d4f8265994182eae453729fd8/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:00,713 INFO [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:00,715 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2 2023-07-24 12:15:00,715 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2 2023-07-24 12:15:00,716 INFO [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f1b51e1d4f8265994182eae453729fd8 columnFamilyName cf2 2023-07-24 12:15:00,720 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/.tmp/cf2/1db249738ce049e3a24141321b907a23 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/1db249738ce049e3a24141321b907a23 2023-07-24 12:15:00,736 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/692579edb5264888aa4103b744a2f08b.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/692579edb5264888aa4103b744a2f08b-top 2023-07-24 12:15:00,739 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 7b2eccad2ce02a648b563be8a6c9f93e/cf1 of 7b2eccad2ce02a648b563be8a6c9f93e into f98742cb66294e5b91193eb16cf37c24(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:00,739 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1652): Completed compaction of 4 (all) file(s) in 7b2eccad2ce02a648b563be8a6c9f93e/cf2 of 7b2eccad2ce02a648b563be8a6c9f93e into 1db249738ce049e3a24141321b907a23(size=5.5 K), total size for store is 5.5 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:00,741 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7b2eccad2ce02a648b563be8a6c9f93e: 2023-07-24 12:15:00,741 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7b2eccad2ce02a648b563be8a6c9f93e: 2023-07-24 12:15:00,741 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e., storeName=7b2eccad2ce02a648b563be8a6c9f93e/cf1, priority=12, startTime=1690200900619; duration=0sec 2023-07-24 12:15:00,741 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e., storeName=7b2eccad2ce02a648b563be8a6c9f93e/cf2, priority=12, startTime=1690200900619; duration=0sec 2023-07-24 12:15:00,742 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:00,742 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:00,746 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/9a102cd162f249a087b7a5984ccba105.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/9a102cd162f249a087b7a5984ccba105-top 2023-07-24 12:15:00,752 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/a406428afd6848e1ac9573741b2f2872.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a406428afd6848e1ac9573741b2f2872-top 2023-07-24 12:15:00,761 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/c42ea765e2b24588a4a836cf8f33d6b0.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/c42ea765e2b24588a4a836cf8f33d6b0-top 2023-07-24 12:15:00,767 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-6bb994da2b534b8c959238f2707401dc 2023-07-24 12:15:00,771 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-a80050507df24ed78b174bd165a5a930 2023-07-24 12:15:00,778 DEBUG [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-bbb0688273a74cdf80bec7200cbc5879 2023-07-24 12:15:00,778 INFO [StoreOpener-f1b51e1d4f8265994182eae453729fd8-1] regionserver.HStore(310): Store=f1b51e1d4f8265994182eae453729fd8/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:00,779 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:00,781 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:00,783 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionUnevenDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:15:00,785 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:00,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened f1b51e1d4f8265994182eae453729fd8; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71722993, jitterRate=0.06875587999820709}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:15:00,786 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for f1b51e1d4f8265994182eae453729fd8: 2023-07-24 12:15:00,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8., pid=17, masterSystemTime=1690200900500 2023-07-24 12:15:00,787 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:00,788 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:15:00,788 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 12:15:00,788 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:15:00,788 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8. 2023-07-24 12:15:00,788 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1912): f1b51e1d4f8265994182eae453729fd8/cf1 is initiating minor compaction (all files) 2023-07-24 12:15:00,788 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8. 2023-07-24 12:15:00,788 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of f1b51e1d4f8265994182eae453729fd8/cf1 in testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8. 2023-07-24 12:15:00,788 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1912): f1b51e1d4f8265994182eae453729fd8/cf2 is initiating minor compaction (all files) 2023-07-24 12:15:00,788 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegion(2259): Starting compaction of f1b51e1d4f8265994182eae453729fd8/cf2 in testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8. 2023-07-24 12:15:00,789 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/ed220ebfec37496bb6f5efd49e84532e.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ed220ebfec37496bb6f5efd49e84532e-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/a115aba766a9442a8956f79a6f674f2f.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/a115aba766a9442a8956f79a6f674f2f-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/df90e6c0608840eba909e7abe5f3a50a.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/df90e6c0608840eba909e7abe5f3a50a-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-d41a8a9025e246d89f1527a5f8d3faf0, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-3a5e9c1e694d4cf8a1e9226147da4628, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-ea30a72aa0544e16a31b0a0d8774946e] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/.tmp, totalSize=34.9 K 2023-07-24 12:15:00,789 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/692579edb5264888aa4103b744a2f08b.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/692579edb5264888aa4103b744a2f08b-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/9a102cd162f249a087b7a5984ccba105.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/9a102cd162f249a087b7a5984ccba105-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/c42ea765e2b24588a4a836cf8f33d6b0.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/c42ea765e2b24588a4a836cf8f33d6b0-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/a406428afd6848e1ac9573741b2f2872.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a406428afd6848e1ac9573741b2f2872-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-a80050507df24ed78b174bd165a5a930, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-bbb0688273a74cdf80bec7200cbc5879, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-6bb994da2b534b8c959238f2707401dc] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/.tmp, totalSize=34.9 K 2023-07-24 12:15:00,790 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting c7cdcc2a7d3d41a3afba02ba9edc6ec1.f5e0e5aefac66bded47d55c23d50c766, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690200897820 2023-07-24 12:15:00,790 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 692579edb5264888aa4103b744a2f08b.f5e0e5aefac66bded47d55c23d50c766, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690200897820 2023-07-24 12:15:00,792 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8. 2023-07-24 12:15:00,792 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8. 2023-07-24 12:15:00,792 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting ed220ebfec37496bb6f5efd49e84532e.f5e0e5aefac66bded47d55c23d50c766, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690200898188 2023-07-24 12:15:00,792 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 9a102cd162f249a087b7a5984ccba105.f5e0e5aefac66bded47d55c23d50c766, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690200898188 2023-07-24 12:15:00,793 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting a115aba766a9442a8956f79a6f674f2f.f5e0e5aefac66bded47d55c23d50c766, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690200898464 2023-07-24 12:15:00,793 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting c42ea765e2b24588a4a836cf8f33d6b0.f5e0e5aefac66bded47d55c23d50c766, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690200898464 2023-07-24 12:15:00,793 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting df90e6c0608840eba909e7abe5f3a50a.f5e0e5aefac66bded47d55c23d50c766, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690200898709 2023-07-24 12:15:00,793 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting a406428afd6848e1ac9573741b2f2872.f5e0e5aefac66bded47d55c23d50c766, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690200898709 2023-07-24 12:15:00,794 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-d41a8a9025e246d89f1527a5f8d3faf0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690200898967 2023-07-24 12:15:00,794 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-a80050507df24ed78b174bd165a5a930, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690200898967 2023-07-24 12:15:00,794 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-3a5e9c1e694d4cf8a1e9226147da4628, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690200899212 2023-07-24 12:15:00,795 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-bbb0688273a74cdf80bec7200cbc5879, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690200899212 2023-07-24 12:15:00,795 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-ea30a72aa0544e16a31b0a0d8774946e, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690200899446 2023-07-24 12:15:00,795 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-6bb994da2b534b8c959238f2707401dc, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690200899446 2023-07-24 12:15:00,803 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=16 updating hbase:meta row=f1b51e1d4f8265994182eae453729fd8, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:00,803 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690200900803"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200900803"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200900803"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200900803"}]},"ts":"1690200900803"} 2023-07-24 12:15:00,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=17, resume processing ppid=16 2023-07-24 12:15:00,810 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=17, ppid=16, state=SUCCESS; OpenRegionProcedure f1b51e1d4f8265994182eae453729fd8, server=jenkins-hbase6.apache.org,36983,1690200894572 in 459 msec 2023-07-24 12:15:00,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=16, resume processing ppid=12 2023-07-24 12:15:00,815 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=16, ppid=12, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f1b51e1d4f8265994182eae453729fd8, ASSIGN in 619 msec 2023-07-24 12:15:00,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 in 1.3460 sec 2023-07-24 12:15:00,828 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] throttle.PressureAwareThroughputController(145): f1b51e1d4f8265994182eae453729fd8#cf1#compaction#17 average throughput is 0.64 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:00,830 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] throttle.PressureAwareThroughputController(145): f1b51e1d4f8265994182eae453729fd8#cf2#compaction#16 average throughput is 0.32 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:00,856 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/.tmp/cf1/7a793966a2dc449bbc32adcda2de3169 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/7a793966a2dc449bbc32adcda2de3169 2023-07-24 12:15:00,857 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/.tmp/cf2/66c91cb29e0745f3b36611c7a027289f as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/66c91cb29e0745f3b36611c7a027289f 2023-07-24 12:15:00,864 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f1b51e1d4f8265994182eae453729fd8/cf1 of f1b51e1d4f8265994182eae453729fd8 into 7a793966a2dc449bbc32adcda2de3169(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:00,865 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for f1b51e1d4f8265994182eae453729fd8: 2023-07-24 12:15:00,865 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8., storeName=f1b51e1d4f8265994182eae453729fd8/cf1, priority=9, startTime=1690200900787; duration=0sec 2023-07-24 12:15:00,865 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:00,867 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in f1b51e1d4f8265994182eae453729fd8/cf2 of f1b51e1d4f8265994182eae453729fd8 into 66c91cb29e0745f3b36611c7a027289f(size=6.7 K), total size for store is 6.7 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:00,868 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for f1b51e1d4f8265994182eae453729fd8: 2023-07-24 12:15:00,868 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8., storeName=f1b51e1d4f8265994182eae453729fd8/cf2, priority=9, startTime=1690200900788; duration=0sec 2023-07-24 12:15:00,868 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:00,982 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36983] ipc.CallRunner(144): callId: 106 service: ClientService methodName: Scan size: 157 connection: 172.31.2.20:48798 deadline: 1690200960982, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766. is not online on jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:01,101 DEBUG [Listener at localhost/37279] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-07-24 12:15:01,113 INFO [Listener at localhost/37279] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-24 12:15:01,115 DEBUG [Listener at localhost/37279] client.ClientScanner(240): Advancing internal scanner to startKey at '26', inclusive 2023-07-24 12:15:01,151 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:15:01,164 INFO [Listener at localhost/37279] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionUnevenDaughter 2023-07-24 12:15:01,169 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$11(2418): Client=jenkins//172.31.2.20 disable testSplitTableRegionUnevenDaughter 2023-07-24 12:15:01,177 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=19, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:01,182 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200901182"}]},"ts":"1690200901182"} 2023-07-24 12:15:01,184 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLING in hbase:meta 2023-07-24 12:15:01,186 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionUnevenDaughter to state=DISABLING 2023-07-24 12:15:01,188 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7b2eccad2ce02a648b563be8a6c9f93e, UNASSIGN}, {pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f1b51e1d4f8265994182eae453729fd8, UNASSIGN}] 2023-07-24 12:15:01,189 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-07-24 12:15:01,191 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=21, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f1b51e1d4f8265994182eae453729fd8, UNASSIGN 2023-07-24 12:15:01,193 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=f1b51e1d4f8265994182eae453729fd8, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:01,193 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690200901193"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200901193"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200901193"}]},"ts":"1690200901193"} 2023-07-24 12:15:01,195 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=22, ppid=21, state=RUNNABLE; CloseRegionProcedure f1b51e1d4f8265994182eae453729fd8, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:01,199 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=20, ppid=19, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7b2eccad2ce02a648b563be8a6c9f93e, UNASSIGN 2023-07-24 12:15:01,200 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=7b2eccad2ce02a648b563be8a6c9f93e, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:01,201 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690200901200"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200901200"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200901200"}]},"ts":"1690200901200"} 2023-07-24 12:15:01,203 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=23, ppid=20, state=RUNNABLE; CloseRegionProcedure 7b2eccad2ce02a648b563be8a6c9f93e, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:01,298 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-07-24 12:15:01,348 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 7b2eccad2ce02a648b563be8a6c9f93e 2023-07-24 12:15:01,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 7b2eccad2ce02a648b563be8a6c9f93e, disabling compactions & flushes 2023-07-24 12:15:01,349 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e. 2023-07-24 12:15:01,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e. 2023-07-24 12:15:01,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e. after waiting 0 ms 2023-07-24 12:15:01,349 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e. 2023-07-24 12:15:01,355 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/ed220ebfec37496bb6f5efd49e84532e.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ed220ebfec37496bb6f5efd49e84532e-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/a115aba766a9442a8956f79a6f674f2f.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/a115aba766a9442a8956f79a6f674f2f-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/df90e6c0608840eba909e7abe5f3a50a.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/df90e6c0608840eba909e7abe5f3a50a-bottom] to archive 2023-07-24 12:15:01,357 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:15:01,363 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,366 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/ed220ebfec37496bb6f5efd49e84532e.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/ed220ebfec37496bb6f5efd49e84532e.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,368 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/a115aba766a9442a8956f79a6f674f2f.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/a115aba766a9442a8956f79a6f674f2f.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,370 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/df90e6c0608840eba909e7abe5f3a50a.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/df90e6c0608840eba909e7abe5f3a50a.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,396 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/692579edb5264888aa4103b744a2f08b.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/692579edb5264888aa4103b744a2f08b-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/9a102cd162f249a087b7a5984ccba105.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/9a102cd162f249a087b7a5984ccba105-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/c42ea765e2b24588a4a836cf8f33d6b0.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/c42ea765e2b24588a4a836cf8f33d6b0-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/a406428afd6848e1ac9573741b2f2872.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a406428afd6848e1ac9573741b2f2872-bottom] to archive 2023-07-24 12:15:01,398 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:15:01,401 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/692579edb5264888aa4103b744a2f08b.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/692579edb5264888aa4103b744a2f08b.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,404 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/9a102cd162f249a087b7a5984ccba105.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/9a102cd162f249a087b7a5984ccba105.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,406 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/c42ea765e2b24588a4a836cf8f33d6b0.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/c42ea765e2b24588a4a836cf8f33d6b0.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,407 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/a406428afd6848e1ac9573741b2f2872.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/a406428afd6848e1ac9573741b2f2872.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,414 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 12:15:01,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:01,416 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e. 2023-07-24 12:15:01,416 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 7b2eccad2ce02a648b563be8a6c9f93e: 2023-07-24 12:15:01,419 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 7b2eccad2ce02a648b563be8a6c9f93e 2023-07-24 12:15:01,419 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:01,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing f1b51e1d4f8265994182eae453729fd8, disabling compactions & flushes 2023-07-24 12:15:01,419 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8. 2023-07-24 12:15:01,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8. 2023-07-24 12:15:01,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8. after waiting 0 ms 2023-07-24 12:15:01,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8. 2023-07-24 12:15:01,420 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=20 updating hbase:meta row=7b2eccad2ce02a648b563be8a6c9f93e, regionState=CLOSED 2023-07-24 12:15:01,422 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690200901419"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200901419"}]},"ts":"1690200901419"} 2023-07-24 12:15:01,423 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/ed220ebfec37496bb6f5efd49e84532e.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ed220ebfec37496bb6f5efd49e84532e-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/a115aba766a9442a8956f79a6f674f2f.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/a115aba766a9442a8956f79a6f674f2f-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/df90e6c0608840eba909e7abe5f3a50a.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/df90e6c0608840eba909e7abe5f3a50a-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-d41a8a9025e246d89f1527a5f8d3faf0, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-3a5e9c1e694d4cf8a1e9226147da4628, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-ea30a72aa0544e16a31b0a0d8774946e] to archive 2023-07-24 12:15:01,425 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:15:01,428 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=23, resume processing ppid=20 2023-07-24 12:15:01,428 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=23, ppid=20, state=SUCCESS; CloseRegionProcedure 7b2eccad2ce02a648b563be8a6c9f93e, server=jenkins-hbase6.apache.org,36983,1690200894572 in 221 msec 2023-07-24 12:15:01,428 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,430 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=20, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=7b2eccad2ce02a648b563be8a6c9f93e, UNASSIGN in 240 msec 2023-07-24 12:15:01,430 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/ed220ebfec37496bb6f5efd49e84532e.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/ed220ebfec37496bb6f5efd49e84532e.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,432 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/a115aba766a9442a8956f79a6f674f2f.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/a115aba766a9442a8956f79a6f674f2f.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,433 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/df90e6c0608840eba909e7abe5f3a50a.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/df90e6c0608840eba909e7abe5f3a50a.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,435 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-d41a8a9025e246d89f1527a5f8d3faf0 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-d41a8a9025e246d89f1527a5f8d3faf0 2023-07-24 12:15:01,436 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-3a5e9c1e694d4cf8a1e9226147da4628 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-3a5e9c1e694d4cf8a1e9226147da4628 2023-07-24 12:15:01,438 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-ea30a72aa0544e16a31b0a0d8774946e to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-ea30a72aa0544e16a31b0a0d8774946e 2023-07-24 12:15:01,440 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/692579edb5264888aa4103b744a2f08b.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/692579edb5264888aa4103b744a2f08b-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/9a102cd162f249a087b7a5984ccba105.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/9a102cd162f249a087b7a5984ccba105-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/c42ea765e2b24588a4a836cf8f33d6b0.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/c42ea765e2b24588a4a836cf8f33d6b0-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/a406428afd6848e1ac9573741b2f2872.f5e0e5aefac66bded47d55c23d50c766->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a406428afd6848e1ac9573741b2f2872-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-a80050507df24ed78b174bd165a5a930, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-bbb0688273a74cdf80bec7200cbc5879, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-6bb994da2b534b8c959238f2707401dc] to archive 2023-07-24 12:15:01,442 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:15:01,445 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/692579edb5264888aa4103b744a2f08b.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/692579edb5264888aa4103b744a2f08b.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,446 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/9a102cd162f249a087b7a5984ccba105.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/9a102cd162f249a087b7a5984ccba105.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,448 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/c42ea765e2b24588a4a836cf8f33d6b0.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/c42ea765e2b24588a4a836cf8f33d6b0.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,449 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/a406428afd6848e1ac9573741b2f2872.f5e0e5aefac66bded47d55c23d50c766 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/a406428afd6848e1ac9573741b2f2872.f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,451 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-a80050507df24ed78b174bd165a5a930 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-a80050507df24ed78b174bd165a5a930 2023-07-24 12:15:01,453 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-bbb0688273a74cdf80bec7200cbc5879 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-bbb0688273a74cdf80bec7200cbc5879 2023-07-24 12:15:01,454 DEBUG [StoreCloser-testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-6bb994da2b534b8c959238f2707401dc to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/testSplitTableRegionUnevenDaughter=f5e0e5aefac66bded47d55c23d50c766-6bb994da2b534b8c959238f2707401dc 2023-07-24 12:15:01,461 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 12:15:01,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:01,463 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8. 2023-07-24 12:15:01,463 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for f1b51e1d4f8265994182eae453729fd8: 2023-07-24 12:15:01,466 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:01,466 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=21 updating hbase:meta row=f1b51e1d4f8265994182eae453729fd8, regionState=CLOSED 2023-07-24 12:15:01,467 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.","families":{"info":[{"qualifier":"regioninfo","vlen":70,"tag":[],"timestamp":"1690200901466"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200901466"}]},"ts":"1690200901466"} 2023-07-24 12:15:01,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=22, resume processing ppid=21 2023-07-24 12:15:01,471 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=22, ppid=21, state=SUCCESS; CloseRegionProcedure f1b51e1d4f8265994182eae453729fd8, server=jenkins-hbase6.apache.org,36983,1690200894572 in 273 msec 2023-07-24 12:15:01,474 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=21, resume processing ppid=19 2023-07-24 12:15:01,474 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=21, ppid=19, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionUnevenDaughter, region=f1b51e1d4f8265994182eae453729fd8, UNASSIGN in 283 msec 2023-07-24 12:15:01,475 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200901475"}]},"ts":"1690200901475"} 2023-07-24 12:15:01,477 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionUnevenDaughter, state=DISABLED in hbase:meta 2023-07-24 12:15:01,480 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionUnevenDaughter to state=DISABLED 2023-07-24 12:15:01,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter in 310 msec 2023-07-24 12:15:01,500 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=19 2023-07-24 12:15:01,500 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 19 completed 2023-07-24 12:15:01,505 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$5(2228): Client=jenkins//172.31.2.20 delete testSplitTableRegionUnevenDaughter 2023-07-24 12:15:01,511 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:01,514 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=24, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:01,516 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=24, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:01,517 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-24 12:15:01,524 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,524 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e 2023-07-24 12:15:01,524 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:01,530 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/recovered.edits] 2023-07-24 12:15:01,530 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/recovered.edits] 2023-07-24 12:15:01,530 DEBUG [HFileArchiver-4] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/recovered.edits] 2023-07-24 12:15:01,536 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/7a793966a2dc449bbc32adcda2de3169 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf1/7a793966a2dc449bbc32adcda2de3169 2023-07-24 12:15:01,536 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/f98742cb66294e5b91193eb16cf37c24 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf1/f98742cb66294e5b91193eb16cf37c24 2023-07-24 12:15:01,536 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/3a5e9c1e694d4cf8a1e9226147da4628 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/3a5e9c1e694d4cf8a1e9226147da4628 2023-07-24 12:15:01,541 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/a115aba766a9442a8956f79a6f674f2f to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/a115aba766a9442a8956f79a6f674f2f 2023-07-24 12:15:01,542 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/66c91cb29e0745f3b36611c7a027289f to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/cf2/66c91cb29e0745f3b36611c7a027289f 2023-07-24 12:15:01,543 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/1db249738ce049e3a24141321b907a23 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/cf2/1db249738ce049e3a24141321b907a23 2023-07-24 12:15:01,544 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/c7cdcc2a7d3d41a3afba02ba9edc6ec1 2023-07-24 12:15:01,549 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/d41a8a9025e246d89f1527a5f8d3faf0 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/d41a8a9025e246d89f1527a5f8d3faf0 2023-07-24 12:15:01,550 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/recovered.edits/90.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8/recovered.edits/90.seqid 2023-07-24 12:15:01,550 DEBUG [HFileArchiver-4] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/recovered.edits/90.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e/recovered.edits/90.seqid 2023-07-24 12:15:01,551 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:01,551 DEBUG [HFileArchiver-4] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/7b2eccad2ce02a648b563be8a6c9f93e 2023-07-24 12:15:01,552 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/df90e6c0608840eba909e7abe5f3a50a to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/df90e6c0608840eba909e7abe5f3a50a 2023-07-24 12:15:01,553 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ea30a72aa0544e16a31b0a0d8774946e to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ea30a72aa0544e16a31b0a0d8774946e 2023-07-24 12:15:01,555 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ed220ebfec37496bb6f5efd49e84532e to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf1/ed220ebfec37496bb6f5efd49e84532e 2023-07-24 12:15:01,558 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/692579edb5264888aa4103b744a2f08b to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/692579edb5264888aa4103b744a2f08b 2023-07-24 12:15:01,559 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/6bb994da2b534b8c959238f2707401dc to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/6bb994da2b534b8c959238f2707401dc 2023-07-24 12:15:01,561 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/9a102cd162f249a087b7a5984ccba105 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/9a102cd162f249a087b7a5984ccba105 2023-07-24 12:15:01,563 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a406428afd6848e1ac9573741b2f2872 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a406428afd6848e1ac9573741b2f2872 2023-07-24 12:15:01,564 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a80050507df24ed78b174bd165a5a930 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/a80050507df24ed78b174bd165a5a930 2023-07-24 12:15:01,566 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/bbb0688273a74cdf80bec7200cbc5879 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/bbb0688273a74cdf80bec7200cbc5879 2023-07-24 12:15:01,568 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/c42ea765e2b24588a4a836cf8f33d6b0 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/cf2/c42ea765e2b24588a4a836cf8f33d6b0 2023-07-24 12:15:01,571 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/recovered.edits/85.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766/recovered.edits/85.seqid 2023-07-24 12:15:01,572 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionUnevenDaughter/f5e0e5aefac66bded47d55c23d50c766 2023-07-24 12:15:01,572 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionUnevenDaughter regions 2023-07-24 12:15:01,576 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=24, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:01,608 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionUnevenDaughter from hbase:meta 2023-07-24 12:15:01,619 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-24 12:15:01,653 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionUnevenDaughter' descriptor. 2023-07-24 12:15:01,655 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=24, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:01,655 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionUnevenDaughter' from region states. 2023-07-24 12:15:01,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200901655"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:01,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200901655"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:01,655 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200901655"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:01,659 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-24 12:15:01,659 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => f5e0e5aefac66bded47d55c23d50c766, NAME => 'testSplitTableRegionUnevenDaughter,,1690200897307.f5e0e5aefac66bded47d55c23d50c766.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 7b2eccad2ce02a648b563be8a6c9f93e, NAME => 'testSplitTableRegionUnevenDaughter,,1690200899469.7b2eccad2ce02a648b563be8a6c9f93e.', STARTKEY => '', ENDKEY => '26'}, {ENCODED => f1b51e1d4f8265994182eae453729fd8, NAME => 'testSplitTableRegionUnevenDaughter,26,1690200899469.f1b51e1d4f8265994182eae453729fd8.', STARTKEY => '26', ENDKEY => ''}] 2023-07-24 12:15:01,659 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionUnevenDaughter' as deleted. 2023-07-24 12:15:01,659 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionUnevenDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690200901659"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:01,661 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionUnevenDaughter state from META 2023-07-24 12:15:01,663 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=24, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:01,665 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter in 157 msec 2023-07-24 12:15:01,799 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 12:15:01,821 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=24 2023-07-24 12:15:01,822 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionUnevenDaughter, procId: 24 completed 2023-07-24 12:15:01,843 INFO [Listener at localhost/37279] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionUnevenDaughter Thread=396 (was 364) Potentially hanging thread: hconnection-0x3d545144-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:44594 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x595be136-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:39326 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: (jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:51206 [Waiting for operation #12] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:44578 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-2 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x595be136-shared-pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.LinkedBlockingQueue.take(LinkedBlockingQueue.java:442) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:44276 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Timer for 'HBase' metrics system java.lang.Object.wait(Native Method) java.util.TimerThread.mainLoop(Timer.java:552) java.util.TimerThread.run(Timer.java:505) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:51456 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:51224 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:44610 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:44618 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:51280 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-metaLookup-shared--pool-1 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-3 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-metaLookup-shared--pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:44634 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:44612 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:51250 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.SynchronousQueue$TransferStack.awaitFulfill(SynchronousQueue.java:458) java.util.concurrent.SynchronousQueue$TransferStack.transfer(SynchronousQueue.java:362) java.util.concurrent.SynchronousQueue.take(SynchronousQueue.java:924) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:39116 [Waiting for operation #19] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:44168 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-3 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:2;jenkins-hbase6:36983-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RSProcedureDispatcher-pool-2 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:39758 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x595be136-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=876 (was 703) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=314 (was 315), ProcessCount=169 (was 169), AvailableMemoryMB=1693 (was 1760) 2023-07-24 12:15:01,865 INFO [Listener at localhost/37279] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=396, OpenFileDescriptor=876, MaxFileDescriptor=60000, SystemLoadAverage=314, ProcessCount=169, AvailableMemoryMB=1693 2023-07-24 12:15:01,865 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:15:01,870 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 12:15:01,871 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(492): Client=jenkins//172.31.2.20 set balanceSwitch=false 2023-07-24 12:15:01,873 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:01,875 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 12:15:01,876 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 12:15:01,878 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:01,880 DEBUG [HFileArchiver-6] backup.HFileArchiver(153): Directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850 empty. 2023-07-24 12:15:01,881 DEBUG [HFileArchiver-6] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:01,881 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-07-24 12:15:01,884 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:namespace' 2023-07-24 12:15:01,885 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'hbase:meta' 2023-07-24 12:15:01,895 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/.tabledesc/.tableinfo.0000000001 2023-07-24 12:15:01,896 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6ee0f9264c2e2de0329ac96e30326850, NAME => 'testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitWithoutPONR', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp 2023-07-24 12:15:01,907 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:01,907 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1604): Closing 6ee0f9264c2e2de0329ac96e30326850, disabling compactions & flushes 2023-07-24 12:15:01,907 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:01,907 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:01,907 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. after waiting 0 ms 2023-07-24 12:15:01,907 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:01,907 INFO [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:01,907 DEBUG [RegionOpenAndInit-testSplitWithoutPONR-pool-0] regionserver.HRegion(1558): Region close journal for 6ee0f9264c2e2de0329ac96e30326850: 2023-07-24 12:15:01,910 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 12:15:01,911 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690200901911"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200901911"}]},"ts":"1690200901911"} 2023-07-24 12:15:01,914 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 12:15:01,915 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 12:15:01,915 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200901915"}]},"ts":"1690200901915"} 2023-07-24 12:15:01,917 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLING in hbase:meta 2023-07-24 12:15:01,920 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase6.apache.org=0} racks are {/default-rack=0} 2023-07-24 12:15:01,921 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 12:15:01,922 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 12:15:01,922 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 12:15:01,922 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 12:15:01,922 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, ASSIGN}] 2023-07-24 12:15:01,924 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, ASSIGN 2023-07-24 12:15:01,925 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=26, ppid=25, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,36983,1690200894572; forceNewPlan=false, retain=false 2023-07-24 12:15:02,075 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 12:15:02,076 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=6ee0f9264c2e2de0329ac96e30326850, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,077 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690200902076"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200902076"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200902076"}]},"ts":"1690200902076"} 2023-07-24 12:15:02,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=27, ppid=26, state=RUNNABLE; OpenRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:02,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:02,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6ee0f9264c2e2de0329ac96e30326850, NAME => 'testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850.', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:15:02,236 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:02,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:02,236 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:02,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:02,237 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:02,238 INFO [StoreOpener-6ee0f9264c2e2de0329ac96e30326850-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:02,239 DEBUG [StoreOpener-6ee0f9264c2e2de0329ac96e30326850-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1 2023-07-24 12:15:02,240 DEBUG [StoreOpener-6ee0f9264c2e2de0329ac96e30326850-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1 2023-07-24 12:15:02,240 INFO [StoreOpener-6ee0f9264c2e2de0329ac96e30326850-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ee0f9264c2e2de0329ac96e30326850 columnFamilyName cf1 2023-07-24 12:15:02,241 INFO [StoreOpener-6ee0f9264c2e2de0329ac96e30326850-1] regionserver.HStore(310): Store=6ee0f9264c2e2de0329ac96e30326850/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:02,241 INFO [StoreOpener-6ee0f9264c2e2de0329ac96e30326850-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:02,242 DEBUG [StoreOpener-6ee0f9264c2e2de0329ac96e30326850-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2 2023-07-24 12:15:02,242 DEBUG [StoreOpener-6ee0f9264c2e2de0329ac96e30326850-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2 2023-07-24 12:15:02,244 INFO [StoreOpener-6ee0f9264c2e2de0329ac96e30326850-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ee0f9264c2e2de0329ac96e30326850 columnFamilyName cf2 2023-07-24 12:15:02,244 INFO [StoreOpener-6ee0f9264c2e2de0329ac96e30326850-1] regionserver.HStore(310): Store=6ee0f9264c2e2de0329ac96e30326850/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:02,245 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:02,246 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:02,248 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:15:02,250 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:02,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 12:15:02,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 6ee0f9264c2e2de0329ac96e30326850; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60577418, jitterRate=-0.0973261296749115}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:15:02,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 6ee0f9264c2e2de0329ac96e30326850: 2023-07-24 12:15:02,254 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850., pid=27, masterSystemTime=1690200902232 2023-07-24 12:15:02,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:02,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:02,256 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=26 updating hbase:meta row=6ee0f9264c2e2de0329ac96e30326850, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,257 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690200902256"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200902256"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200902256"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200902256"}]},"ts":"1690200902256"} 2023-07-24 12:15:02,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=27, resume processing ppid=26 2023-07-24 12:15:02,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=27, ppid=26, state=SUCCESS; OpenRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 in 179 msec 2023-07-24 12:15:02,263 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=26, resume processing ppid=25 2023-07-24 12:15:02,263 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=26, ppid=25, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, ASSIGN in 339 msec 2023-07-24 12:15:02,264 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 12:15:02,265 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200902265"}]},"ts":"1690200902265"} 2023-07-24 12:15:02,266 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=ENABLED in hbase:meta 2023-07-24 12:15:02,269 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=25, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitWithoutPONR execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 12:15:02,271 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR in 398 msec 2023-07-24 12:15:02,386 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:02,388 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-24 12:15:02,388 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-24 12:15:02,388 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:02,389 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-24 12:15:02,389 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-24 12:15:02,390 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,390 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,391 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,391 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,391 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,391 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,391 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,391 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,392 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,392 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:02,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,392 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,392 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:02,392 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,393 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:02,393 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:02,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,393 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:02,393 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:02,393 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:02,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:02,393 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:02,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,394 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:02,394 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:02,393 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:02,394 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:02,394 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:02,394 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:02,394 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:02,396 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:02,394 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:02,396 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:02,397 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:02,397 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:02,397 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:02,397 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:02,397 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:02,397 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:02,397 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:02,397 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. started... 2023-07-24 12:15:02,398 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ee0f9264c2e2de0329ac96e30326850 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 12:15:02,398 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,399 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,399 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,399 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:02,399 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,400 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:02,414 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf1/02d229cf681148aba9236a36b8ebd770 2023-07-24 12:15:02,430 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf2/fd9ae77842c94456af6a1f3b4a8f7110 2023-07-24 12:15:02,436 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf1/02d229cf681148aba9236a36b8ebd770 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/02d229cf681148aba9236a36b8ebd770 2023-07-24 12:15:02,442 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/02d229cf681148aba9236a36b8ebd770, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:15:02,442 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf2/fd9ae77842c94456af6a1f3b4a8f7110 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/fd9ae77842c94456af6a1f3b4a8f7110 2023-07-24 12:15:02,448 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/fd9ae77842c94456af6a1f3b4a8f7110, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:15:02,449 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 6ee0f9264c2e2de0329ac96e30326850 in 51ms, sequenceid=6, compaction requested=false 2023-07-24 12:15:02,450 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitWithoutPONR' 2023-07-24 12:15:02,450 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ee0f9264c2e2de0329ac96e30326850: 2023-07-24 12:15:02,450 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:02,450 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:02,450 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:02,450 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,450 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:02,450 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:02,453 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,453 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:02,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:02,453 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,453 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:02,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-24 12:15:02,456 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@fc31be5[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:02,456 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-24 12:15:02,456 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,457 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,457 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,457 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,457 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,457 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:02,457 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:02,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,457 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:02,457 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:02,458 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:02,458 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:02,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,457 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:02,458 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:02,458 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:02,458 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:02,459 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:02,459 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,459 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:02,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:02,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:02,459 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:02,460 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:02,460 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:02,460 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:02,460 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:02,460 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:02,461 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:02,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:02,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:02,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:02,463 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:15:02,465 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-24 12:15:02,465 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-24 12:15:02,465 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:02,465 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:02,465 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-24 12:15:02,465 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:02,472 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,472 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,472 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,472 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,472 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,472 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:02,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:02,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,472 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,472 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:02,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,473 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,473 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,474 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:02,474 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,474 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,475 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:02,476 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,476 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,476 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,476 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,477 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,479 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,479 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,479 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,479 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,479 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,479 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,479 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,479 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,479 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-24 12:15:02,480 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:02,479 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,479 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,479 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:02,480 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,480 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,480 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,480 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,479 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,481 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,481 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:02,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,481 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,481 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,481 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,481 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:02,481 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,481 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,481 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,481 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,581 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:02,583 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:02,601 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:02,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:02,603 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-24 12:15:02,603 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-24 12:15:02,603 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:02,604 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-24 12:15:02,604 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-24 12:15:02,604 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,604 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,605 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,605 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,605 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,605 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,605 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,606 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,606 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,606 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,607 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:02,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:02,607 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:02,608 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:02,608 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:02,608 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:02,608 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:02,608 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:02,608 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:02,608 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:02,609 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:02,609 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:02,609 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:02,609 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:02,609 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:02,609 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:02,610 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:02,610 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:02,610 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:02,610 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:02,610 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:02,610 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:02,610 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:02,610 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:02,610 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. started... 2023-07-24 12:15:02,610 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:02,611 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ee0f9264c2e2de0329ac96e30326850 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:02,612 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,618 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,618 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,618 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:02,618 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,619 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:02,629 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf1/83bad71e47904701b2c0952293f52c92 2023-07-24 12:15:02,645 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf2/85dbf47ba56b46e99eed2f4714e30876 2023-07-24 12:15:02,650 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf1/83bad71e47904701b2c0952293f52c92 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/83bad71e47904701b2c0952293f52c92 2023-07-24 12:15:02,655 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/83bad71e47904701b2c0952293f52c92, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:15:02,656 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf2/85dbf47ba56b46e99eed2f4714e30876 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/85dbf47ba56b46e99eed2f4714e30876 2023-07-24 12:15:02,661 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/85dbf47ba56b46e99eed2f4714e30876, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:15:02,662 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ee0f9264c2e2de0329ac96e30326850 in 52ms, sequenceid=19, compaction requested=false 2023-07-24 12:15:02,662 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ee0f9264c2e2de0329ac96e30326850: 2023-07-24 12:15:02,662 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:02,662 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:02,662 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:02,662 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,662 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:02,662 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:02,664 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,664 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:02,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:02,664 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,664 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:02,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:02,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:02,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,666 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:02,666 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-24 12:15:02,666 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5429374b[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:02,666 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-24 12:15:02,666 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,667 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,667 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,667 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,668 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,668 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,668 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:02,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,668 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:02,668 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:02,668 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:02,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,668 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:02,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,668 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:02,668 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:02,668 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:02,668 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:02,668 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:02,669 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:02,670 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:02,670 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,670 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:02,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:02,671 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:02,673 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:02,673 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:02,673 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:02,673 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:02,673 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:02,674 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:02,674 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:02,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:02,676 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:02,677 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,678 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,679 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:15:02,679 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-24 12:15:02,679 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-24 12:15:02,679 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:02,679 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:02,679 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-24 12:15:02,679 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:02,681 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,681 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,681 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,681 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,681 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,681 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,681 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:02,681 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,682 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:02,681 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,682 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,682 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:02,682 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,683 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,683 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:02,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,685 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:02,685 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,686 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,689 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,689 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,689 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,689 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,689 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,689 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,689 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,689 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,690 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,690 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,690 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,690 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,690 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,690 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,690 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,690 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,690 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:02,690 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-24 12:15:02,690 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:02,690 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,690 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,690 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,691 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:02,691 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:02,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,691 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,791 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:02,792 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:02,810 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:02,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:02,812 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-24 12:15:02,812 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-24 12:15:02,812 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:02,813 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-24 12:15:02,813 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-24 12:15:02,814 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,814 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,816 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,816 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,816 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,816 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,816 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,817 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:02,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,817 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,817 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:02,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,818 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:02,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:02,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:02,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:02,818 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:02,818 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:02,818 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:02,818 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:02,818 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:02,818 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:02,819 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:02,819 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:02,819 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:02,820 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:02,820 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:02,820 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:02,820 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:02,820 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:02,820 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:02,821 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:02,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:02,821 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:02,821 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:02,821 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:02,822 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:02,822 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:02,822 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:02,822 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. started... 2023-07-24 12:15:02,822 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ee0f9264c2e2de0329ac96e30326850 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:02,823 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,823 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,824 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,824 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:02,824 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,824 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:02,838 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf1/027e47b648a94df287297f2e2eaee0e8 2023-07-24 12:15:02,856 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf2/7b8195fd9108416a9d39279bd41705b0 2023-07-24 12:15:02,862 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf1/027e47b648a94df287297f2e2eaee0e8 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/027e47b648a94df287297f2e2eaee0e8 2023-07-24 12:15:02,867 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/027e47b648a94df287297f2e2eaee0e8, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:15:02,868 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf2/7b8195fd9108416a9d39279bd41705b0 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/7b8195fd9108416a9d39279bd41705b0 2023-07-24 12:15:02,873 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/7b8195fd9108416a9d39279bd41705b0, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:15:02,874 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ee0f9264c2e2de0329ac96e30326850 in 52ms, sequenceid=32, compaction requested=true 2023-07-24 12:15:02,875 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ee0f9264c2e2de0329ac96e30326850: 2023-07-24 12:15:02,875 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:02,875 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:02,875 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:02,875 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,875 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:02,875 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:02,877 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,877 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:02,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:02,877 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,877 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:02,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:02,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:02,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:02,879 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-24 12:15:02,880 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@8e96a0b[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:02,880 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-24 12:15:02,880 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,881 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,881 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,881 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,882 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:02,882 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:02,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,882 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:02,882 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:02,882 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:02,882 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,882 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,882 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:02,882 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,882 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:02,882 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:02,882 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:02,883 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:02,884 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:02,884 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:02,884 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:02,884 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:02,884 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:02,884 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,884 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:02,884 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:02,885 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:02,885 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:02,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:02,885 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:02,885 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:02,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,886 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:02,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,887 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,888 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:15:02,888 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-24 12:15:02,888 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-24 12:15:02,888 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:02,888 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:02,888 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-24 12:15:02,888 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:02,891 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,891 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,891 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,891 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,891 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,891 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,891 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,891 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:02,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:02,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,891 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,893 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:02,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,896 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,896 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,897 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:02,897 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,898 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,899 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:02,899 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,899 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:02,900 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,900 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,903 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,903 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,903 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,903 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,903 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:02,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,903 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,904 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,903 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,903 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:02,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,903 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:02,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-24 12:15:02,904 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:02,904 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,904 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:02,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:02,904 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:02,904 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:02,904 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,904 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:02,904 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:02,904 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:02,904 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:02,904 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:02,905 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:02,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:02,905 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:02,905 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,005 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:03,006 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:03,025 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:03,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:03,027 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-24 12:15:03,027 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-24 12:15:03,027 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:03,028 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-24 12:15:03,028 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-24 12:15:03,029 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,029 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,030 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,030 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,030 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,030 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,030 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,031 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,031 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,032 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:03,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,032 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:03,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,032 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:03,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:03,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:03,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,032 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:03,032 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:03,032 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:03,032 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:03,032 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:03,033 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:03,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:03,033 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:03,033 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:03,034 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:03,034 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:03,034 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:03,034 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:03,033 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:03,034 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:03,034 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:03,034 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:03,034 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:03,036 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:03,034 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:03,036 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:03,036 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:03,036 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:03,036 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:03,037 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. started... 2023-07-24 12:15:03,037 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ee0f9264c2e2de0329ac96e30326850 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:03,041 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,042 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,042 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,042 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,042 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:03,042 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:03,049 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf1/cac876a46ff841fdb348394bce2ae945 2023-07-24 12:15:03,064 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf2/cdd0fec6dd2e4ff19fe6c988461e518d 2023-07-24 12:15:03,070 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf1/cac876a46ff841fdb348394bce2ae945 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/cac876a46ff841fdb348394bce2ae945 2023-07-24 12:15:03,076 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/cac876a46ff841fdb348394bce2ae945, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:15:03,077 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf2/cdd0fec6dd2e4ff19fe6c988461e518d as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/cdd0fec6dd2e4ff19fe6c988461e518d 2023-07-24 12:15:03,082 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/cdd0fec6dd2e4ff19fe6c988461e518d, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:15:03,084 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ee0f9264c2e2de0329ac96e30326850 in 47ms, sequenceid=45, compaction requested=true 2023-07-24 12:15:03,084 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ee0f9264c2e2de0329ac96e30326850: 2023-07-24 12:15:03,084 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:03,084 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:03,084 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:03,084 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,084 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:03,084 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:03,086 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,086 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:03,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:03,086 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,087 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:03,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:03,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:03,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:03,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-24 12:15:03,088 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@402166cd[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:03,088 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-24 12:15:03,089 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,090 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,090 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,090 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,090 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:03,090 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,090 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:03,090 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:03,090 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,090 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:03,090 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:03,090 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:03,090 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:03,090 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:03,091 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:03,091 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:03,092 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:03,092 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,092 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:03,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:03,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:03,092 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:03,094 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:03,094 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:03,094 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:03,094 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:03,094 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:03,094 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:03,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:03,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:15:03,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-24 12:15:03,098 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-24 12:15:03,098 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:03,098 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:03,099 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-24 12:15:03,099 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:03,100 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,100 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,100 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:03,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:03,100 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,100 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,101 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:03,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,102 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:03,102 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,104 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:03,104 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,105 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,108 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,108 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,108 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,108 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,108 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,108 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,108 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:03,108 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,108 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-24 12:15:03,109 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:03,108 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,108 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,108 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,108 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,109 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,109 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,109 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:03,109 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,110 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,110 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:03,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,109 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,110 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,110 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,110 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,110 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,110 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,210 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:03,211 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:03,230 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:03,232 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:03,233 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-24 12:15:03,233 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-24 12:15:03,233 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:03,233 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-24 12:15:03,233 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-24 12:15:03,233 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,233 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,235 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,235 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,235 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,235 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,235 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,235 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:03,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,236 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,236 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,236 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:03,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,236 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:03,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:03,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,236 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:03,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:03,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,236 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:03,237 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:03,236 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:03,237 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:03,237 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:03,237 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:03,237 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:03,237 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:03,237 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:03,237 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:03,237 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:03,237 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:03,237 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:03,237 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:03,237 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:03,237 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:03,237 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. started... 2023-07-24 12:15:03,237 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:03,237 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:03,237 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:03,237 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:03,238 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ee0f9264c2e2de0329ac96e30326850 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:03,237 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:03,238 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:03,238 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:03,239 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,240 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,240 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:03,240 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,240 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,240 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:03,253 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf1/19aa0444a70f4aa4b98a9546577f1616 2023-07-24 12:15:03,273 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf2/d2fb26c0d3414f36897dcd6ff784515f 2023-07-24 12:15:03,279 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf1/19aa0444a70f4aa4b98a9546577f1616 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/19aa0444a70f4aa4b98a9546577f1616 2023-07-24 12:15:03,283 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/19aa0444a70f4aa4b98a9546577f1616, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:15:03,284 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf2/d2fb26c0d3414f36897dcd6ff784515f as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/d2fb26c0d3414f36897dcd6ff784515f 2023-07-24 12:15:03,289 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/d2fb26c0d3414f36897dcd6ff784515f, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:15:03,291 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ee0f9264c2e2de0329ac96e30326850 in 53ms, sequenceid=58, compaction requested=true 2023-07-24 12:15:03,291 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ee0f9264c2e2de0329ac96e30326850: 2023-07-24 12:15:03,291 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:03,291 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:03,291 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:03,291 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,291 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:03,291 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:03,293 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,293 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:03,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:03,294 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,294 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:03,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:03,294 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:03,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,295 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,296 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:03,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-24 12:15:03,296 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@57aa411f[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:03,296 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-24 12:15:03,296 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,297 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,297 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,297 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,297 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,298 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:03,297 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,297 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,297 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:03,298 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:03,298 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:03,298 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:03,298 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:03,298 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:03,298 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:03,298 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:03,298 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:03,299 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:03,300 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:03,300 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:03,300 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:03,300 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:03,300 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,300 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:03,301 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:03,302 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:03,302 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:03,302 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:03,303 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:03,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:03,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,303 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:03,304 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,305 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,306 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:15:03,306 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-24 12:15:03,306 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-24 12:15:03,306 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:03,306 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:03,306 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-24 12:15:03,306 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:03,307 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,308 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,308 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,308 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:03,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:03,308 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,308 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,308 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,308 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:03,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,308 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,308 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,309 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,309 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,309 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:03,309 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,310 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:03,311 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,311 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,312 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,312 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,312 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,317 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,317 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,317 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,317 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,317 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-24 12:15:03,317 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:03,317 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,317 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,317 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,317 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:03,317 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,318 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,318 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,318 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,318 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,318 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,318 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,318 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,317 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,317 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,318 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,318 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,318 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:03,319 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,319 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,319 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:03,419 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:03,421 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:03,440 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:03,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitWithoutPONR was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:03,443 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitWithoutPONR 2023-07-24 12:15:03,443 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitWithoutPONR' 2023-07-24 12:15:03,443 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:03,443 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitWithoutPONR' starting 'acquire' 2023-07-24 12:15:03,443 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitWithoutPONR', kicking off acquire phase on members. 2023-07-24 12:15:03,444 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,444 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,446 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,446 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,446 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,446 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,446 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,446 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:03,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,446 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,447 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:03,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,447 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:03,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:03,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:03,448 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:03,448 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:03,448 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:03,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,448 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:03,448 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:03,448 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitWithoutPONR 2023-07-24 12:15:03,448 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:03,448 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:03,448 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:03,448 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:03,448 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitWithoutPONR is still present, removing 2023-07-24 12:15:03,448 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:03,448 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:03,448 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:03,448 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitWithoutPONR 2023-07-24 12:15:03,449 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:03,449 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitWithoutPONR' with timeout 60000ms 2023-07-24 12:15:03,449 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:03,449 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:03,449 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:03,449 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitWithoutPONR' starting 'acquire' stage 2023-07-24 12:15:03,449 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:03,449 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:03,449 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. started... 2023-07-24 12:15:03,449 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:03,449 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 6ee0f9264c2e2de0329ac96e30326850 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:03,450 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,451 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,451 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,451 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:03,452 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,452 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:03,466 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf1/4cb78d8805614413b005ff50ce5707f3 2023-07-24 12:15:03,474 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-24 12:15:03,475 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint Metrics about HBase RegionObservers 2023-07-24 12:15:03,475 DEBUG [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(130): Registering adapter for the MetricRegistry: RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:03,475 INFO [HBase-Metrics2-1] impl.GlobalMetricRegistriesAdapter(134): Registering RegionServer,sub=Coprocessor.Region.CP_org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro Metrics about HBase RegionObservers 2023-07-24 12:15:03,489 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf2/5af1b0465e95470f87fb547508c6b568 2023-07-24 12:15:03,495 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf1/4cb78d8805614413b005ff50ce5707f3 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/4cb78d8805614413b005ff50ce5707f3 2023-07-24 12:15:03,502 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/4cb78d8805614413b005ff50ce5707f3, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:15:03,503 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf2/5af1b0465e95470f87fb547508c6b568 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5af1b0465e95470f87fb547508c6b568 2023-07-24 12:15:03,509 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5af1b0465e95470f87fb547508c6b568, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:15:03,510 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 6ee0f9264c2e2de0329ac96e30326850 in 61ms, sequenceid=71, compaction requested=true 2023-07-24 12:15:03,510 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 6ee0f9264c2e2de0329ac96e30326850: 2023-07-24 12:15:03,510 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:03,510 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:03,510 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:03,510 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,510 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitWithoutPONR' locally acquired 2023-07-24 12:15:03,510 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitWithoutPONR) in zk 2023-07-24 12:15:03,512 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,512 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:03,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:03,513 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,513 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitWithoutPONR' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:03,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:03,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:03,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:03,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitWithoutPONR' on coordinator 2023-07-24 12:15:03,515 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4aa4a7ce[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:03,515 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitWithoutPONR' starting 'in-barrier' execution. 2023-07-24 12:15:03,515 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,517 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,517 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,517 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,517 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,518 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:03,518 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,518 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,518 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:03,518 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,518 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:03,518 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:03,518 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:03,518 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitWithoutPONR' received 'reached' from coordinator. 2023-07-24 12:15:03,518 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:03,518 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitWithoutPONR' locally completed 2023-07-24 12:15:03,518 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:03,518 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitWithoutPONR' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:03,520 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,520 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:03,520 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:03,520 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitWithoutPONR' has notified controller of completion 2023-07-24 12:15:03,520 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:03,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:03,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:03,520 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:03,520 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:03,520 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:03,520 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:03,520 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitWithoutPONR' completed. 2023-07-24 12:15:03,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:03,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:03,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:03,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitWithoutPONR' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:15:03,524 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitWithoutPONR', counting down latch. Waiting for 0 more 2023-07-24 12:15:03,524 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitWithoutPONR' execution completed 2023-07-24 12:15:03,524 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:03,524 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:03,524 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitWithoutPONR 2023-07-24 12:15:03,524 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitWithoutPONRincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:03,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,527 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,528 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:03,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:03,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,528 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,528 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:03,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:03,529 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,530 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,530 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,530 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,531 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:03,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitWithoutPONR 2023-07-24 12:15:03,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,534 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,534 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,534 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:03,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,534 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,534 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,534 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,534 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitWithoutPONR' 2023-07-24 12:15:03,534 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:03,534 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,534 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:03,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:03,534 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:03,535 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitWithoutPONR'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:03,534 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,535 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,534 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:03,535 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,535 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitWithoutPONR 2023-07-24 12:15:03,535 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:03,535 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:03,535 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,536 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:03,536 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,536 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:03,536 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:03,536 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitWithoutPONR 2023-07-24 12:15:03,536 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitWithoutPONR 2023-07-24 12:15:03,636 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:03,637 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:03,899 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-07-24 12:15:03,901 DEBUG [Listener at localhost/37279] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:03,902 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:03,904 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:03,904 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:03,904 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:03,905 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:03,905 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:03,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:03,905 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:04,153 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,154 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:04,154 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:04,154 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:04,156 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-1,5,PEWorkerGroup] Thread[HFileArchiver-2,5,PEWorkerGroup] Thread[HFileArchiver-3,5,PEWorkerGroup] Thread[HFileArchiver-4,5,PEWorkerGroup] Thread[HFileArchiver-5,5,PEWorkerGroup] Thread[HFileArchiver-6,5,PEWorkerGroup] 2023-07-24 12:15:04,157 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:04,157 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:04,157 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:04,157 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:04,158 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:04,160 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-07-24 12:15:04,162 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:04,162 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:04,163 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:04,163 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:04,163 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:04,164 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:04,164 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:04,164 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:04,165 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:04,165 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,165 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:04,165 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,165 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,165 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,166 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-24 12:15:04,168 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:04,168 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:04,171 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200934171 2023-07-24 12:15:04,173 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,173 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:04,177 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:04,178 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=6ee0f9264c2e2de0329ac96e30326850, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,36983,1690200894572, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572, openSeqNum=2 2023-07-24 12:15:04,178 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:04,178 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200964178 2023-07-24 12:15:04,178 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201024178 2023-07-24 12:15:04,178 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 12:15:04,178 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:04,181 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:04,181 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,181 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:04,181 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:04,428 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,428 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:04,428 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:04,428 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:04,429 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:04,430 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:04,430 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:04,430 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:04,430 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:04,431 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:04,432 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-24 12:15:04,433 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:04,434 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:04,434 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:04,434 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:04,435 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:04,435 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:04,435 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:04,435 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:04,436 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:04,436 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,436 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:04,436 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,436 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,437 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,437 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 12:15:04,437 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:04,438 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:04,438 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200934438 2023-07-24 12:15:04,439 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:04,440 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,444 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:04,444 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=6ee0f9264c2e2de0329ac96e30326850, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,36983,1690200894572, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572, openSeqNum=2 2023-07-24 12:15:04,444 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:04,444 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200964444 2023-07-24 12:15:04,444 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201024444 2023-07-24 12:15:04,444 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 12:15:04,444 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:04,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN}] 2023-07-24 12:15:04,447 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:04,447 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,447 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:04,447 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:04,694 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,695 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:04,695 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:04,695 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:04,696 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:04,696 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:04,696 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:04,696 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:04,697 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:04,697 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:04,699 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-24 12:15:04,699 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:04,700 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:04,700 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:04,700 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:04,701 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:04,701 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:04,701 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:04,702 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:04,702 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:04,702 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,702 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:04,702 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,703 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,703 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,703 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 12:15:04,703 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:04,704 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:04,704 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200934704 2023-07-24 12:15:04,705 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:04,706 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,709 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:04,710 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=6ee0f9264c2e2de0329ac96e30326850, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,36983,1690200894572, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572, openSeqNum=2 2023-07-24 12:15:04,710 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:04,710 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200964710 2023-07-24 12:15:04,710 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201024710 2023-07-24 12:15:04,710 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-07-24 12:15:04,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN}] 2023-07-24 12:15:04,711 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:04,713 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:04,714 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6ee0f9264c2e2de0329ac96e30326850, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:04,714 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690200904714"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200904714"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200904714"}]},"ts":"1690200904714"} 2023-07-24 12:15:04,716 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:04,716 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:04,716 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:04,716 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:04,716 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:04,961 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,961 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:04,961 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:04,961 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:04,962 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:04,962 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:04,963 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:04,963 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:04,963 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:04,964 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:04,966 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 12:15:04,966 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-24 12:15:04,966 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:04,967 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:04,967 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:04,967 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:04,968 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:04,968 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:04,969 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:04,969 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:04,969 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:04,971 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,971 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:04,971 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:04,971 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,971 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,971 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:04,971 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:04,972 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:04,972 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-24 12:15:04,972 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:04,973 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:04,973 INFO [Listener at localhost/37279] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850 to restore RIT 2023-07-24 12:15:04,974 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200934974 2023-07-24 12:15:04,975 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:04,975 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:04,978 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:04,978 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=6ee0f9264c2e2de0329ac96e30326850, regionState=CLOSING, lastHost=jenkins-hbase6.apache.org,36983,1690200894572, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572, openSeqNum=2 2023-07-24 12:15:04,978 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:04,978 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200964978 2023-07-24 12:15:04,979 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201024979 2023-07-24 12:15:04,979 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:04,979 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6ee0f9264c2e2de0329ac96e30326850, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:04,979 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690200904979"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200904979"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200904979"}]},"ts":"1690200904979"} 2023-07-24 12:15:04,981 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:04,981 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:05,134 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 6ee0f9264c2e2de0329ac96e30326850, disabling compactions & flushes 2023-07-24 12:15:05,134 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:05,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:05,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. after waiting 0 ms 2023-07-24 12:15:05,134 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:05,135 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2745): Flushing 6ee0f9264c2e2de0329ac96e30326850 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-24 12:15:05,145 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf1/5e85b49d5be04b3384ed9f9c73792891 2023-07-24 12:15:05,163 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf2/5e88410c0eae4f57b7ed17b12ea51124 2023-07-24 12:15:05,169 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf1/5e85b49d5be04b3384ed9f9c73792891 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/5e85b49d5be04b3384ed9f9c73792891 2023-07-24 12:15:05,174 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/5e85b49d5be04b3384ed9f9c73792891, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 12:15:05,175 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/.tmp/cf2/5e88410c0eae4f57b7ed17b12ea51124 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5e88410c0eae4f57b7ed17b12ea51124 2023-07-24 12:15:05,180 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5e88410c0eae4f57b7ed17b12ea51124, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 12:15:05,181 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 6ee0f9264c2e2de0329ac96e30326850 in 47ms, sequenceid=82, compaction requested=true 2023-07-24 12:15:05,202 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-24 12:15:05,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:05,203 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. 2023-07-24 12:15:05,203 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 6ee0f9264c2e2de0329ac96e30326850: 2023-07-24 12:15:05,206 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,206 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6ee0f9264c2e2de0329ac96e30326850, regionState=CLOSED 2023-07-24 12:15:05,206 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690200905206"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200905206"}]},"ts":"1690200905206"} 2023-07-24 12:15:05,208 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:05,208 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:05,208 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:05,208 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=30, ppid=29, state=SUCCESS, locked=true; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:05,229 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,229 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:05,229 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:05,229 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:05,230 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:05,230 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:05,230 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:05,230 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:05,230 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:05,231 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:05,232 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 12:15:05,232 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-24 12:15:05,232 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-07-24 12:15:05,233 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:05,233 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:05,233 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:05,233 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:05,234 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:05,234 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:05,234 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:05,235 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:05,235 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:05,236 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,236 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:05,236 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:05,236 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:05,236 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,236 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,236 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,236 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:05,236 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:05,236 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:05,236 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 12:15:05,237 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:05,237 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:05,238 INFO [Listener at localhost/37279] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850 to restore RIT 2023-07-24 12:15:05,238 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200935238 2023-07-24 12:15:05,239 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=30, ppid=29, state=RUNNABLE; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:05,239 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:05,241 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:05,242 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=6ee0f9264c2e2de0329ac96e30326850, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,36983,1690200894572, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572, openSeqNum=2 2023-07-24 12:15:05,242 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:05,242 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200965242 2023-07-24 12:15:05,242 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201025242 2023-07-24 12:15:05,242 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:05,242 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=29 updating hbase:meta row=6ee0f9264c2e2de0329ac96e30326850, regionState=CLOSED, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:05,242 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690200905242"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200905242"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200905242"}]},"ts":"1690200905242"} 2023-07-24 12:15:05,244 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:05,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=30, resume processing ppid=29 2023-07-24 12:15:05,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 in 263 msec 2023-07-24 12:15:05,247 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:05,247 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:05,247 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:05,247 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=29, ppid=28, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:05,492 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,492 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:05,493 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:05,493 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:05,493 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:05,493 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:05,493 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:05,493 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:05,493 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:05,494 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:05,495 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 12:15:05,495 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-24 12:15:05,495 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:15:05,496 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:05,496 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:05,497 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:05,497 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:05,497 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:05,497 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:05,498 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:05,498 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:05,498 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:05,499 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,499 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:05,499 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:05,499 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:05,499 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,499 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,499 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,499 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:05,499 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:05,500 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 12:15:05,500 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:05,501 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:05,501 INFO [Listener at localhost/37279] assignment.AssignmentManager(305): Attach pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN to state=OFFLINE, location=null, table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850 to restore RIT 2023-07-24 12:15:05,501 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200935501 2023-07-24 12:15:05,502 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:05,502 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:05,505 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:05,505 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=6ee0f9264c2e2de0329ac96e30326850, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,36983,1690200894572, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572, openSeqNum=2 2023-07-24 12:15:05,505 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:05,505 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200965505 2023-07-24 12:15:05,505 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201025505 2023-07-24 12:15:05,506 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:05,506 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=29, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:05,508 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:05,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=29, resume processing ppid=28 2023-07-24 12:15:05,511 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN in 797 msec 2023-07-24 12:15:05,514 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:05,514 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,514 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:05,514 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:05,756 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,756 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:05,756 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:05,756 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:05,756 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:05,756 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:05,757 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:05,757 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:05,757 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:05,757 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:05,759 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 12:15:05,759 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:15:05,759 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:15:05,759 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:05,759 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:05,760 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:05,760 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:05,760 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:05,760 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:05,761 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:05,761 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:05,761 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:05,762 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,762 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:05,762 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:05,762 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:05,762 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:05,762 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:05,762 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 12:15:05,763 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:05,763 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:05,764 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200935764 2023-07-24 12:15:05,764 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:05,765 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,768 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:05,768 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=6ee0f9264c2e2de0329ac96e30326850, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,36983,1690200894572, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572, openSeqNum=2 2023-07-24 12:15:05,768 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:05,768 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200965768 2023-07-24 12:15:05,768 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201025768 2023-07-24 12:15:05,768 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:05,769 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,769 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,769 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:05,774 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:05,784 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=6ee0f9264c2e2de0329ac96e30326850, threads=14 2023-07-24 12:15:05,785 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5af1b0465e95470f87fb547508c6b568 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,785 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5e88410c0eae4f57b7ed17b12ea51124 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,786 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/7b8195fd9108416a9d39279bd41705b0 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,786 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/85dbf47ba56b46e99eed2f4714e30876 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,786 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/cdd0fec6dd2e4ff19fe6c988461e518d for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,787 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/d2fb26c0d3414f36897dcd6ff784515f for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,787 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/fd9ae77842c94456af6a1f3b4a8f7110 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,788 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/027e47b648a94df287297f2e2eaee0e8 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,789 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/02d229cf681148aba9236a36b8ebd770 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,791 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/19aa0444a70f4aa4b98a9546577f1616 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,791 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/4cb78d8805614413b005ff50ce5707f3 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,792 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/5e85b49d5be04b3384ed9f9c73792891 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,792 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/83bad71e47904701b2c0952293f52c92 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,797 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/cac876a46ff841fdb348394bce2ae945 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:05,971 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/d2fb26c0d3414f36897dcd6ff784515f for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,000 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/cdd0fec6dd2e4ff19fe6c988461e518d for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,003 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5e88410c0eae4f57b7ed17b12ea51124 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,010 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/cac876a46ff841fdb348394bce2ae945 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,015 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/85dbf47ba56b46e99eed2f4714e30876 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,018 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5af1b0465e95470f87fb547508c6b568 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,019 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/4cb78d8805614413b005ff50ce5707f3 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,032 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/7b8195fd9108416a9d39279bd41705b0 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,036 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/fd9ae77842c94456af6a1f3b4a8f7110 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,039 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/027e47b648a94df287297f2e2eaee0e8 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,046 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/02d229cf681148aba9236a36b8ebd770 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,049 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/19aa0444a70f4aa4b98a9546577f1616 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,051 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/83bad71e47904701b2c0952293f52c92 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,424 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/5e85b49d5be04b3384ed9f9c73792891 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,424 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 6ee0f9264c2e2de0329ac96e30326850 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 12:15:06,464 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:06,464 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:06,464 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:06,464 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:06,519 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:06,519 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:06,519 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:06,519 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:06,519 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:06,520 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:06,520 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:06,520 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:06,520 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:06,521 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:06,522 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-24 12:15:06,522 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:15:06,522 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:15:06,522 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:06,523 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:06,523 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:06,523 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:06,524 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:06,524 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:06,525 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:06,525 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:06,526 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:06,527 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:06,527 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:06,527 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:06,527 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:06,527 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:06,527 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:06,527 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:06,527 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:06,527 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:06,527 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-24 12:15:06,528 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:06,528 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:06,529 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200936529 2023-07-24 12:15:06,531 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:06,531 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:06,534 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:06,534 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=6ee0f9264c2e2de0329ac96e30326850, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,36983,1690200894572, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572, openSeqNum=2 2023-07-24 12:15:06,534 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:06,534 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200966534 2023-07-24 12:15:06,534 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201026534 2023-07-24 12:15:06,534 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:06,545 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=28 splitting 14 storefiles, region=6ee0f9264c2e2de0329ac96e30326850, threads=14 2023-07-24 12:15:06,546 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5af1b0465e95470f87fb547508c6b568 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,548 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5e88410c0eae4f57b7ed17b12ea51124 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,549 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/7b8195fd9108416a9d39279bd41705b0 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,549 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/85dbf47ba56b46e99eed2f4714e30876 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,549 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/cdd0fec6dd2e4ff19fe6c988461e518d for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,551 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/d2fb26c0d3414f36897dcd6ff784515f for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,551 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/fd9ae77842c94456af6a1f3b4a8f7110 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,555 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/027e47b648a94df287297f2e2eaee0e8 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,555 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/02d229cf681148aba9236a36b8ebd770 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,556 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/19aa0444a70f4aa4b98a9546577f1616 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,556 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/4cb78d8805614413b005ff50ce5707f3 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,556 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/5e85b49d5be04b3384ed9f9c73792891 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,557 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/83bad71e47904701b2c0952293f52c92 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,562 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=28 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/cac876a46ff841fdb348394bce2ae945 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,731 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/d2fb26c0d3414f36897dcd6ff784515f for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,736 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5af1b0465e95470f87fb547508c6b568 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,739 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/4cb78d8805614413b005ff50ce5707f3 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,743 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/fd9ae77842c94456af6a1f3b4a8f7110 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,759 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/cac876a46ff841fdb348394bce2ae945 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,795 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/19aa0444a70f4aa4b98a9546577f1616 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,796 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/02d229cf681148aba9236a36b8ebd770 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,811 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/027e47b648a94df287297f2e2eaee0e8 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,812 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/5e85b49d5be04b3384ed9f9c73792891 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,814 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/7b8195fd9108416a9d39279bd41705b0 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,814 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5e88410c0eae4f57b7ed17b12ea51124 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,816 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/83bad71e47904701b2c0952293f52c92 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,817 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/cdd0fec6dd2e4ff19fe6c988461e518d for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,818 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=28 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/85dbf47ba56b46e99eed2f4714e30876 for region: 6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:06,818 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=28 split storefiles for region 6ee0f9264c2e2de0329ac96e30326850 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 12:15:06,862 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:06,868 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 12:15:06,870 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 12:15:06,871 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:06,871 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:06,871 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:06,871 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:06,918 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 12:15:07,035 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,035 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:07,035 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:07,035 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:07,035 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:07,036 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:07,036 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:07,036 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:07,036 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:07,037 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:07,038 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-24 12:15:07,038 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:15:07,038 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:15:07,038 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:07,038 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:07,039 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:07,039 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:07,039 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:07,039 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:07,040 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:07,040 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:07,040 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:07,041 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,041 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:07,041 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:07,041 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:07,041 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,041 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,041 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,041 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:07,041 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:07,041 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 12:15:07,042 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:07,043 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:07,043 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200937043 2023-07-24 12:15:07,045 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:07,045 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,048 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:07,048 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=6ee0f9264c2e2de0329ac96e30326850, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,36983,1690200894572, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572, openSeqNum=2 2023-07-24 12:15:07,048 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:07,048 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200967048 2023-07-24 12:15:07,049 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201027049 2023-07-24 12:15:07,049 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:07,052 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:07,055 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:07,055 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,055 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:07,055 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:07,299 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:15:07,299 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:07,299 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:07,299 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:07,300 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:07,300 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:07,300 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:07,300 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:07,301 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:07,301 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:07,302 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=28, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-24 12:15:07,303 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=29, ppid=28, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:15:07,303 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=30, ppid=29, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:15:07,303 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:07,303 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:07,303 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:07,304 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:07,304 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:07,304 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:07,304 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:07,305 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:07,305 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:07,305 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,305 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN 2023-07-24 12:15:07,306 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:07,306 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:07,306 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,306 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,306 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,306 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=29, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=6ee0f9264c2e2de0329ac96e30326850, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:07,306 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=30, ppid=29, state=SUCCESS; CloseRegionProcedure 6ee0f9264c2e2de0329ac96e30326850, server=jenkins-hbase6.apache.org,36983,1690200894572 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:07,306 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 12:15:07,307 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:07,307 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:07,307 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200937307 2023-07-24 12:15:07,308 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:07,309 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=28, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,311 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:07,311 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=6ee0f9264c2e2de0329ac96e30326850, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,36983,1690200894572, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572, openSeqNum=2 2023-07-24 12:15:07,311 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:07,311 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200967311 2023-07-24 12:15:07,311 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201027311 2023-07-24 12:15:07,311 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:07,315 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690200907314"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1690200907314"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1690200907314"}]},"ts":"1690200907314"} 2023-07-24 12:15:07,315 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690200907314"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200907314"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200907314"}]},"ts":"1690200907314"} 2023-07-24 12:15:07,315 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690200907314"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200907314"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200907314"}]},"ts":"1690200907314"} 2023-07-24 12:15:07,320 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8f319344d8b0142d3e6f98cf87650169, ASSIGN}, {pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=509695ae5946d9f03c670f89496b9662, ASSIGN}] 2023-07-24 12:15:07,321 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=509695ae5946d9f03c670f89496b9662, ASSIGN 2023-07-24 12:15:07,322 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=32, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=509695ae5946d9f03c670f89496b9662, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase6.apache.org,36983,1690200894572; forceNewPlan=false, retain=false 2023-07-24 12:15:07,322 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8f319344d8b0142d3e6f98cf87650169, ASSIGN 2023-07-24 12:15:07,323 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=31, ppid=28, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8f319344d8b0142d3e6f98cf87650169, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase6.apache.org,36983,1690200894572; forceNewPlan=false, retain=false 2023-07-24 12:15:07,472 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-24 12:15:07,474 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=8f319344d8b0142d3e6f98cf87650169, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:07,474 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690200907474"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200907474"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200907474"}]},"ts":"1690200907474"} 2023-07-24 12:15:07,476 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=33, ppid=31, state=RUNNABLE; OpenRegionProcedure 8f319344d8b0142d3e6f98cf87650169, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:07,479 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=509695ae5946d9f03c670f89496b9662, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:07,479 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690200907479"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200907479"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200907479"}]},"ts":"1690200907479"} 2023-07-24 12:15:07,480 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=34, ppid=32, state=RUNNABLE; OpenRegionProcedure 509695ae5946d9f03c670f89496b9662, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:07,631 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662. 2023-07-24 12:15:07,631 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 509695ae5946d9f03c670f89496b9662, NAME => 'testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.', STARTKEY => '41', ENDKEY => ''} 2023-07-24 12:15:07,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:07,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:07,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,632 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,633 INFO [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,634 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1 2023-07-24 12:15:07,634 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1 2023-07-24 12:15:07,635 INFO [StoreOpener-509695ae5946d9f03c670f89496b9662-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 509695ae5946d9f03c670f89496b9662 columnFamilyName cf1 2023-07-24 12:15:07,651 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/027e47b648a94df287297f2e2eaee0e8-top 2023-07-24 12:15:07,655 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/02d229cf681148aba9236a36b8ebd770-top 2023-07-24 12:15:07,660 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/19aa0444a70f4aa4b98a9546577f1616-top 2023-07-24 12:15:07,665 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/4cb78d8805614413b005ff50ce5707f3-top 2023-07-24 12:15:07,669 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/5e85b49d5be04b3384ed9f9c73792891-top 2023-07-24 12:15:07,673 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/83bad71e47904701b2c0952293f52c92-top 2023-07-24 12:15:07,678 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/cac876a46ff841fdb348394bce2ae945-top 2023-07-24 12:15:07,678 INFO [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(310): Store=509695ae5946d9f03c670f89496b9662/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:07,678 INFO [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,679 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2 2023-07-24 12:15:07,679 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2 2023-07-24 12:15:07,679 INFO [StoreOpener-509695ae5946d9f03c670f89496b9662-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 509695ae5946d9f03c670f89496b9662 columnFamilyName cf2 2023-07-24 12:15:07,693 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5af1b0465e95470f87fb547508c6b568-top 2023-07-24 12:15:07,697 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5e88410c0eae4f57b7ed17b12ea51124-top 2023-07-24 12:15:07,701 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/7b8195fd9108416a9d39279bd41705b0-top 2023-07-24 12:15:07,705 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/85dbf47ba56b46e99eed2f4714e30876-top 2023-07-24 12:15:07,710 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/cdd0fec6dd2e4ff19fe6c988461e518d-top 2023-07-24 12:15:07,714 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/d2fb26c0d3414f36897dcd6ff784515f-top 2023-07-24 12:15:07,718 DEBUG [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/fd9ae77842c94456af6a1f3b4a8f7110-top 2023-07-24 12:15:07,718 INFO [StoreOpener-509695ae5946d9f03c670f89496b9662-1] regionserver.HStore(310): Store=509695ae5946d9f03c670f89496b9662/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:07,719 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,720 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,722 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:15:07,723 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:07,724 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 509695ae5946d9f03c670f89496b9662; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59816393, jitterRate=-0.10866628587245941}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:15:07,724 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 509695ae5946d9f03c670f89496b9662: 2023-07-24 12:15:07,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662., pid=34, masterSystemTime=1690200907628 2023-07-24 12:15:07,725 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:07,726 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 12:15:07,726 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:15:07,726 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:15:07,727 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662. 2023-07-24 12:15:07,727 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1912): 509695ae5946d9f03c670f89496b9662/cf1 is initiating minor compaction (all files) 2023-07-24 12:15:07,727 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662. 2023-07-24 12:15:07,727 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 509695ae5946d9f03c670f89496b9662/cf1 in testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662. 2023-07-24 12:15:07,727 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1912): 509695ae5946d9f03c670f89496b9662/cf2 is initiating minor compaction (all files) 2023-07-24 12:15:07,727 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 509695ae5946d9f03c670f89496b9662/cf2 in testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662. 2023-07-24 12:15:07,727 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/02d229cf681148aba9236a36b8ebd770-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/83bad71e47904701b2c0952293f52c92-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/027e47b648a94df287297f2e2eaee0e8-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/cac876a46ff841fdb348394bce2ae945-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/19aa0444a70f4aa4b98a9546577f1616-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/4cb78d8805614413b005ff50ce5707f3-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/5e85b49d5be04b3384ed9f9c73792891-top] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/.tmp, totalSize=34.9 K 2023-07-24 12:15:07,728 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/fd9ae77842c94456af6a1f3b4a8f7110-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/85dbf47ba56b46e99eed2f4714e30876-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/7b8195fd9108416a9d39279bd41705b0-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/cdd0fec6dd2e4ff19fe6c988461e518d-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/d2fb26c0d3414f36897dcd6ff784515f-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5af1b0465e95470f87fb547508c6b568-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5e88410c0eae4f57b7ed17b12ea51124-top] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/.tmp, totalSize=34.9 K 2023-07-24 12:15:07,728 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690200902378 2023-07-24 12:15:07,729 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690200902584 2023-07-24 12:15:07,729 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690200902378 2023-07-24 12:15:07,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662. 2023-07-24 12:15:07,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662. 2023-07-24 12:15:07,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169. 2023-07-24 12:15:07,729 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8f319344d8b0142d3e6f98cf87650169, NAME => 'testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.', STARTKEY => '', ENDKEY => '41'} 2023-07-24 12:15:07,729 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690200902793 2023-07-24 12:15:07,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:07,730 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690200902584 2023-07-24 12:15:07,730 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690200903007 2023-07-24 12:15:07,730 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=32 updating hbase:meta row=509695ae5946d9f03c670f89496b9662, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:07,730 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690200907730"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200907730"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200907730"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200907730"}]},"ts":"1690200907730"} 2023-07-24 12:15:07,730 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690200902793 2023-07-24 12:15:07,730 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitWithoutPONR 8f319344d8b0142d3e6f98cf87650169 2023-07-24 12:15:07,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:07,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 8f319344d8b0142d3e6f98cf87650169 2023-07-24 12:15:07,731 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 8f319344d8b0142d3e6f98cf87650169 2023-07-24 12:15:07,731 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690200903007 2023-07-24 12:15:07,730 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1690200903212 2023-07-24 12:15:07,732 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1690200903422 2023-07-24 12:15:07,732 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1690200903212 2023-07-24 12:15:07,733 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690200903638 2023-07-24 12:15:07,733 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1690200903422 2023-07-24 12:15:07,734 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690200903638 2023-07-24 12:15:07,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=34, resume processing ppid=32 2023-07-24 12:15:07,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=34, ppid=32, state=SUCCESS; OpenRegionProcedure 509695ae5946d9f03c670f89496b9662, server=jenkins-hbase6.apache.org,36983,1690200894572 in 252 msec 2023-07-24 12:15:07,736 INFO [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8f319344d8b0142d3e6f98cf87650169 2023-07-24 12:15:07,737 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=32, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=509695ae5946d9f03c670f89496b9662, ASSIGN in 417 msec 2023-07-24 12:15:07,738 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1 2023-07-24 12:15:07,738 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1 2023-07-24 12:15:07,738 INFO [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8f319344d8b0142d3e6f98cf87650169 columnFamilyName cf1 2023-07-24 12:15:07,758 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] throttle.PressureAwareThroughputController(145): 509695ae5946d9f03c670f89496b9662#cf1#compaction#32 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:07,765 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] throttle.PressureAwareThroughputController(145): 509695ae5946d9f03c670f89496b9662#cf2#compaction#33 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:07,776 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/027e47b648a94df287297f2e2eaee0e8-bottom 2023-07-24 12:15:07,786 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/02d229cf681148aba9236a36b8ebd770-bottom 2023-07-24 12:15:07,790 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/.tmp/cf1/994c89ad7ba54cfaa4f8dd38814f8c3f as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/994c89ad7ba54cfaa4f8dd38814f8c3f 2023-07-24 12:15:07,792 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/19aa0444a70f4aa4b98a9546577f1616-bottom 2023-07-24 12:15:07,804 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 509695ae5946d9f03c670f89496b9662/cf1 of 509695ae5946d9f03c670f89496b9662 into 994c89ad7ba54cfaa4f8dd38814f8c3f(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:07,804 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 509695ae5946d9f03c670f89496b9662: 2023-07-24 12:15:07,804 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662., storeName=509695ae5946d9f03c670f89496b9662/cf1, priority=9, startTime=1690200907725; duration=0sec 2023-07-24 12:15:07,804 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:07,805 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/4cb78d8805614413b005ff50ce5707f3-bottom 2023-07-24 12:15:07,810 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/5e85b49d5be04b3384ed9f9c73792891-bottom 2023-07-24 12:15:07,811 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/.tmp/cf2/b8f3592ed72e4dd0b33a3407b27621a4 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/b8f3592ed72e4dd0b33a3407b27621a4 2023-07-24 12:15:07,815 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/83bad71e47904701b2c0952293f52c92-bottom 2023-07-24 12:15:07,818 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 509695ae5946d9f03c670f89496b9662/cf2 of 509695ae5946d9f03c670f89496b9662 into b8f3592ed72e4dd0b33a3407b27621a4(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:07,818 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 509695ae5946d9f03c670f89496b9662: 2023-07-24 12:15:07,818 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662., storeName=509695ae5946d9f03c670f89496b9662/cf2, priority=9, startTime=1690200907726; duration=0sec 2023-07-24 12:15:07,818 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:07,819 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/cac876a46ff841fdb348394bce2ae945-bottom 2023-07-24 12:15:07,819 INFO [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(310): Store=8f319344d8b0142d3e6f98cf87650169/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:07,820 INFO [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8f319344d8b0142d3e6f98cf87650169 2023-07-24 12:15:07,820 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2 2023-07-24 12:15:07,821 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2 2023-07-24 12:15:07,821 INFO [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8f319344d8b0142d3e6f98cf87650169 columnFamilyName cf2 2023-07-24 12:15:07,836 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5af1b0465e95470f87fb547508c6b568-bottom 2023-07-24 12:15:07,839 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5e88410c0eae4f57b7ed17b12ea51124-bottom 2023-07-24 12:15:07,844 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/7b8195fd9108416a9d39279bd41705b0-bottom 2023-07-24 12:15:07,848 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/85dbf47ba56b46e99eed2f4714e30876-bottom 2023-07-24 12:15:07,853 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/cdd0fec6dd2e4ff19fe6c988461e518d-bottom 2023-07-24 12:15:07,858 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/d2fb26c0d3414f36897dcd6ff784515f-bottom 2023-07-24 12:15:07,861 DEBUG [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/fd9ae77842c94456af6a1f3b4a8f7110-bottom 2023-07-24 12:15:07,862 INFO [StoreOpener-8f319344d8b0142d3e6f98cf87650169-1] regionserver.HStore(310): Store=8f319344d8b0142d3e6f98cf87650169/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:07,862 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169 2023-07-24 12:15:07,863 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169 2023-07-24 12:15:07,865 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitWithoutPONR descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:15:07,866 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 8f319344d8b0142d3e6f98cf87650169 2023-07-24 12:15:07,867 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 8f319344d8b0142d3e6f98cf87650169; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=58801472, jitterRate=-0.12378978729248047}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:15:07,867 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 8f319344d8b0142d3e6f98cf87650169: 2023-07-24 12:15:07,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169., pid=33, masterSystemTime=1690200907628 2023-07-24 12:15:07,868 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:07,869 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 12:15:07,869 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:15:07,869 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:15:07,869 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169. 2023-07-24 12:15:07,869 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169. 2023-07-24 12:15:07,869 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1912): 8f319344d8b0142d3e6f98cf87650169/cf1 is initiating minor compaction (all files) 2023-07-24 12:15:07,869 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1912): 8f319344d8b0142d3e6f98cf87650169/cf2 is initiating minor compaction (all files) 2023-07-24 12:15:07,869 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8f319344d8b0142d3e6f98cf87650169/cf1 in testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169. 2023-07-24 12:15:07,869 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 8f319344d8b0142d3e6f98cf87650169/cf2 in testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169. 2023-07-24 12:15:07,870 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/fd9ae77842c94456af6a1f3b4a8f7110-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/85dbf47ba56b46e99eed2f4714e30876-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/7b8195fd9108416a9d39279bd41705b0-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/cdd0fec6dd2e4ff19fe6c988461e518d-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/d2fb26c0d3414f36897dcd6ff784515f-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5af1b0465e95470f87fb547508c6b568-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5e88410c0eae4f57b7ed17b12ea51124-bottom] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/.tmp, totalSize=34.9 K 2023-07-24 12:15:07,870 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/02d229cf681148aba9236a36b8ebd770-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/83bad71e47904701b2c0952293f52c92-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/027e47b648a94df287297f2e2eaee0e8-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/cac876a46ff841fdb348394bce2ae945-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/19aa0444a70f4aa4b98a9546577f1616-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/4cb78d8805614413b005ff50ce5707f3-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/5e85b49d5be04b3384ed9f9c73792891-bottom] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/.tmp, totalSize=34.9 K 2023-07-24 12:15:07,870 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690200902378 2023-07-24 12:15:07,870 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169. 2023-07-24 12:15:07,870 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690200902378 2023-07-24 12:15:07,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169. 2023-07-24 12:15:07,871 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690200902584 2023-07-24 12:15:07,871 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690200902584 2023-07-24 12:15:07,871 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=31 updating hbase:meta row=8f319344d8b0142d3e6f98cf87650169, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:07,871 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690200902793 2023-07-24 12:15:07,871 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690200907871"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200907871"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200907871"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200907871"}]},"ts":"1690200907871"} 2023-07-24 12:15:07,872 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690200902793 2023-07-24 12:15:07,872 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690200903007 2023-07-24 12:15:07,872 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690200903007 2023-07-24 12:15:07,873 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690200903212 2023-07-24 12:15:07,873 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690200903212 2023-07-24 12:15:07,873 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690200903422 2023-07-24 12:15:07,873 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690200903422 2023-07-24 12:15:07,874 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690200903638 2023-07-24 12:15:07,874 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690200903638 2023-07-24 12:15:07,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=33, resume processing ppid=31 2023-07-24 12:15:07,876 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=33, ppid=31, state=SUCCESS; OpenRegionProcedure 8f319344d8b0142d3e6f98cf87650169, server=jenkins-hbase6.apache.org,36983,1690200894572 in 397 msec 2023-07-24 12:15:07,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=31, resume processing ppid=28 2023-07-24 12:15:07,878 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=31, ppid=28, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8f319344d8b0142d3e6f98cf87650169, ASSIGN in 558 msec 2023-07-24 12:15:07,880 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 in 3.9780 sec 2023-07-24 12:15:07,889 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] throttle.PressureAwareThroughputController(145): 8f319344d8b0142d3e6f98cf87650169#cf2#compaction#34 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:07,890 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8f319344d8b0142d3e6f98cf87650169#cf1#compaction#35 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:07,905 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/.tmp/cf1/0469de54e18144c4930c8e2712f787a6 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/0469de54e18144c4930c8e2712f787a6 2023-07-24 12:15:07,910 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8f319344d8b0142d3e6f98cf87650169/cf1 of 8f319344d8b0142d3e6f98cf87650169 into 0469de54e18144c4930c8e2712f787a6(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:07,910 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8f319344d8b0142d3e6f98cf87650169: 2023-07-24 12:15:07,910 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169., storeName=8f319344d8b0142d3e6f98cf87650169/cf1, priority=9, startTime=1690200907868; duration=0sec 2023-07-24 12:15:07,910 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:08,063 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36983] ipc.CallRunner(144): callId: 239 service: ClientService methodName: Scan size: 143 connection: 172.31.2.20:48798 deadline: 1690200968062, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850. is not online on jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:08,173 DEBUG [Listener at localhost/37279] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-24 12:15:08,182 INFO [Listener at localhost/37279] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-24 12:15:08,184 DEBUG [Listener at localhost/37279] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-24 12:15:08,213 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:15:08,214 INFO [Listener at localhost/37279] client.HBaseAdmin$15(890): Started disable of testSplitWithoutPONR 2023-07-24 12:15:08,215 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$11(2418): Client=jenkins//172.31.2.20 disable testSplitWithoutPONR 2023-07-24 12:15:08,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=35, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:08,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-07-24 12:15:08,219 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200908219"}]},"ts":"1690200908219"} 2023-07-24 12:15:08,220 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLING in hbase:meta 2023-07-24 12:15:08,223 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitWithoutPONR to state=DISABLING 2023-07-24 12:15:08,224 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8f319344d8b0142d3e6f98cf87650169, UNASSIGN}, {pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=509695ae5946d9f03c670f89496b9662, UNASSIGN}] 2023-07-24 12:15:08,224 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=37, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=509695ae5946d9f03c670f89496b9662, UNASSIGN 2023-07-24 12:15:08,225 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=509695ae5946d9f03c670f89496b9662, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:08,225 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690200908225"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200908225"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200908225"}]},"ts":"1690200908225"} 2023-07-24 12:15:08,227 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=38, ppid=37, state=RUNNABLE; CloseRegionProcedure 509695ae5946d9f03c670f89496b9662, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:08,229 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=36, ppid=35, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8f319344d8b0142d3e6f98cf87650169, UNASSIGN 2023-07-24 12:15:08,230 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=8f319344d8b0142d3e6f98cf87650169, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:08,230 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690200908230"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200908230"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200908230"}]},"ts":"1690200908230"} 2023-07-24 12:15:08,231 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=39, ppid=36, state=RUNNABLE; CloseRegionProcedure 8f319344d8b0142d3e6f98cf87650169, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:08,306 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/.tmp/cf2/599ac29ab5ea472481743dea3e783878 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/599ac29ab5ea472481743dea3e783878 2023-07-24 12:15:08,311 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8f319344d8b0142d3e6f98cf87650169/cf2 of 8f319344d8b0142d3e6f98cf87650169 into 599ac29ab5ea472481743dea3e783878(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:08,311 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8f319344d8b0142d3e6f98cf87650169: 2023-07-24 12:15:08,311 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169., storeName=8f319344d8b0142d3e6f98cf87650169/cf2, priority=9, startTime=1690200907869; duration=0sec 2023-07-24 12:15:08,311 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:08,319 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-07-24 12:15:08,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 8f319344d8b0142d3e6f98cf87650169 2023-07-24 12:15:08,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 8f319344d8b0142d3e6f98cf87650169, disabling compactions & flushes 2023-07-24 12:15:08,379 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169. 2023-07-24 12:15:08,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169. 2023-07-24 12:15:08,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169. after waiting 0 ms 2023-07-24 12:15:08,379 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169. 2023-07-24 12:15:08,390 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/02d229cf681148aba9236a36b8ebd770-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/83bad71e47904701b2c0952293f52c92-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/027e47b648a94df287297f2e2eaee0e8-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/cac876a46ff841fdb348394bce2ae945-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/19aa0444a70f4aa4b98a9546577f1616-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/4cb78d8805614413b005ff50ce5707f3-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/5e85b49d5be04b3384ed9f9c73792891-bottom] to archive 2023-07-24 12:15:08,391 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:15:08,393 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,394 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,396 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,397 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,399 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,400 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,401 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,403 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/fd9ae77842c94456af6a1f3b4a8f7110-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/85dbf47ba56b46e99eed2f4714e30876-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/7b8195fd9108416a9d39279bd41705b0-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/cdd0fec6dd2e4ff19fe6c988461e518d-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/d2fb26c0d3414f36897dcd6ff784515f-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5af1b0465e95470f87fb547508c6b568-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5e88410c0eae4f57b7ed17b12ea51124-bottom] to archive 2023-07-24 12:15:08,404 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:15:08,406 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,407 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,408 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,410 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,411 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,412 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,413 DEBUG [StoreCloser-testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 12:15:08,417 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:08,418 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169. 2023-07-24 12:15:08,418 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 8f319344d8b0142d3e6f98cf87650169: 2023-07-24 12:15:08,419 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 8f319344d8b0142d3e6f98cf87650169 2023-07-24 12:15:08,419 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:08,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 509695ae5946d9f03c670f89496b9662, disabling compactions & flushes 2023-07-24 12:15:08,419 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662. 2023-07-24 12:15:08,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662. 2023-07-24 12:15:08,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662. after waiting 0 ms 2023-07-24 12:15:08,419 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662. 2023-07-24 12:15:08,420 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=36 updating hbase:meta row=8f319344d8b0142d3e6f98cf87650169, regionState=CLOSED 2023-07-24 12:15:08,420 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690200908420"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200908420"}]},"ts":"1690200908420"} 2023-07-24 12:15:08,421 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/02d229cf681148aba9236a36b8ebd770-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/83bad71e47904701b2c0952293f52c92-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/027e47b648a94df287297f2e2eaee0e8-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/cac876a46ff841fdb348394bce2ae945-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/19aa0444a70f4aa4b98a9546577f1616-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/4cb78d8805614413b005ff50ce5707f3-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/5e85b49d5be04b3384ed9f9c73792891-top] to archive 2023-07-24 12:15:08,421 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:15:08,424 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=39, resume processing ppid=36 2023-07-24 12:15:08,424 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=39, ppid=36, state=SUCCESS; CloseRegionProcedure 8f319344d8b0142d3e6f98cf87650169, server=jenkins-hbase6.apache.org,36983,1690200894572 in 190 msec 2023-07-24 12:15:08,425 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=36, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=8f319344d8b0142d3e6f98cf87650169, UNASSIGN in 200 msec 2023-07-24 12:15:08,427 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,429 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,430 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,431 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,433 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,434 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,435 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,437 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/fd9ae77842c94456af6a1f3b4a8f7110-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/85dbf47ba56b46e99eed2f4714e30876-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/7b8195fd9108416a9d39279bd41705b0-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/cdd0fec6dd2e4ff19fe6c988461e518d-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/d2fb26c0d3414f36897dcd6ff784515f-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5af1b0465e95470f87fb547508c6b568-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5e88410c0eae4f57b7ed17b12ea51124-top] to archive 2023-07-24 12:15:08,437 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:15:08,439 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,440 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,441 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,442 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,443 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,444 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,445 DEBUG [StoreCloser-testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 12:15:08,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:08,449 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662. 2023-07-24 12:15:08,449 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 509695ae5946d9f03c670f89496b9662: 2023-07-24 12:15:08,451 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:08,451 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=37 updating hbase:meta row=509695ae5946d9f03c670f89496b9662, regionState=CLOSED 2023-07-24 12:15:08,451 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690200908451"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200908451"}]},"ts":"1690200908451"} 2023-07-24 12:15:08,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=38, resume processing ppid=37 2023-07-24 12:15:08,455 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=38, ppid=37, state=SUCCESS; CloseRegionProcedure 509695ae5946d9f03c670f89496b9662, server=jenkins-hbase6.apache.org,36983,1690200894572 in 226 msec 2023-07-24 12:15:08,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=37, resume processing ppid=35 2023-07-24 12:15:08,457 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=37, ppid=35, state=SUCCESS; TransitRegionStateProcedure table=testSplitWithoutPONR, region=509695ae5946d9f03c670f89496b9662, UNASSIGN in 231 msec 2023-07-24 12:15:08,458 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200908457"}]},"ts":"1690200908457"} 2023-07-24 12:15:08,459 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitWithoutPONR, state=DISABLED in hbase:meta 2023-07-24 12:15:08,460 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitWithoutPONR to state=DISABLED 2023-07-24 12:15:08,463 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR in 246 msec 2023-07-24 12:15:08,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=35 2023-07-24 12:15:08,521 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitWithoutPONR, procId: 35 completed 2023-07-24 12:15:08,522 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$5(2228): Client=jenkins//172.31.2.20 delete testSplitWithoutPONR 2023-07-24 12:15:08,523 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:08,524 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=40, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:08,525 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-07-24 12:15:08,525 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=40, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:08,532 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,532 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:08,532 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169 2023-07-24 12:15:08,534 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/recovered.edits] 2023-07-24 12:15:08,535 DEBUG [HFileArchiver-1] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/recovered.edits] 2023-07-24 12:15:08,535 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/recovered.edits] 2023-07-24 12:15:08,539 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/027e47b648a94df287297f2e2eaee0e8 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/027e47b648a94df287297f2e2eaee0e8 2023-07-24 12:15:08,539 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/.regioninfo to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/.regioninfo 2023-07-24 12:15:08,540 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/.regioninfo to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/.regioninfo 2023-07-24 12:15:08,543 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf1/027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf1/027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,543 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/02d229cf681148aba9236a36b8ebd770 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/02d229cf681148aba9236a36b8ebd770 2023-07-24 12:15:08,545 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf1/02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf1/02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,545 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/19aa0444a70f4aa4b98a9546577f1616 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/19aa0444a70f4aa4b98a9546577f1616 2023-07-24 12:15:08,546 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf1/027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf1/027e47b648a94df287297f2e2eaee0e8.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,546 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf1/19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf1/19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,546 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/4cb78d8805614413b005ff50ce5707f3 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/4cb78d8805614413b005ff50ce5707f3 2023-07-24 12:15:08,547 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf1/02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf1/02d229cf681148aba9236a36b8ebd770.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,548 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf1/4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf1/4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,548 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/5e85b49d5be04b3384ed9f9c73792891 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/5e85b49d5be04b3384ed9f9c73792891 2023-07-24 12:15:08,549 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf1/19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf1/19aa0444a70f4aa4b98a9546577f1616.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,549 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf1/5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf1/5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,550 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/83bad71e47904701b2c0952293f52c92 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/83bad71e47904701b2c0952293f52c92 2023-07-24 12:15:08,551 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf1/83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf1/83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,551 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf1/4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf1/4cb78d8805614413b005ff50ce5707f3.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,552 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/cac876a46ff841fdb348394bce2ae945 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf1/cac876a46ff841fdb348394bce2ae945 2023-07-24 12:15:08,552 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf1/cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf1/cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,552 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf1/5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf1/5e85b49d5be04b3384ed9f9c73792891.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,554 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf1/83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf1/83bad71e47904701b2c0952293f52c92.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,556 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf1/cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf1/cac876a46ff841fdb348394bce2ae945.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,556 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf2/5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf2/5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,556 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5af1b0465e95470f87fb547508c6b568 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5af1b0465e95470f87fb547508c6b568 2023-07-24 12:15:08,558 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf2/5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf2/5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,559 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5e88410c0eae4f57b7ed17b12ea51124 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/5e88410c0eae4f57b7ed17b12ea51124 2023-07-24 12:15:08,560 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf2/7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf2/7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,561 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/7b8195fd9108416a9d39279bd41705b0 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/7b8195fd9108416a9d39279bd41705b0 2023-07-24 12:15:08,561 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf2/5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf2/5af1b0465e95470f87fb547508c6b568.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,562 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf2/85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf2/85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,563 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/85dbf47ba56b46e99eed2f4714e30876 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/85dbf47ba56b46e99eed2f4714e30876 2023-07-24 12:15:08,564 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf2/5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf2/5e88410c0eae4f57b7ed17b12ea51124.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,565 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf2/cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf2/cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,566 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/cdd0fec6dd2e4ff19fe6c988461e518d to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/cdd0fec6dd2e4ff19fe6c988461e518d 2023-07-24 12:15:08,566 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf2/7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf2/7b8195fd9108416a9d39279bd41705b0.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,568 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/d2fb26c0d3414f36897dcd6ff784515f to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/d2fb26c0d3414f36897dcd6ff784515f 2023-07-24 12:15:08,569 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf2/d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf2/d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,569 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf2/85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf2/85dbf47ba56b46e99eed2f4714e30876.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,571 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf2/fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/509695ae5946d9f03c670f89496b9662/cf2/fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,571 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/fd9ae77842c94456af6a1f3b4a8f7110 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/cf2/fd9ae77842c94456af6a1f3b4a8f7110 2023-07-24 12:15:08,572 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf2/cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf2/cdd0fec6dd2e4ff19fe6c988461e518d.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,574 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf2/d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf2/d2fb26c0d3414f36897dcd6ff784515f.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,576 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf2/fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/8f319344d8b0142d3e6f98cf87650169/cf2/fd9ae77842c94456af6a1f3b4a8f7110.6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,576 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/994c89ad7ba54cfaa4f8dd38814f8c3f to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf1/994c89ad7ba54cfaa4f8dd38814f8c3f 2023-07-24 12:15:08,577 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/recovered.edits/85.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850/recovered.edits/85.seqid 2023-07-24 12:15:08,578 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/6ee0f9264c2e2de0329ac96e30326850 2023-07-24 12:15:08,579 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/0469de54e18144c4930c8e2712f787a6 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf1/0469de54e18144c4930c8e2712f787a6 2023-07-24 12:15:08,580 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/b8f3592ed72e4dd0b33a3407b27621a4 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/cf2/b8f3592ed72e4dd0b33a3407b27621a4 2023-07-24 12:15:08,582 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/599ac29ab5ea472481743dea3e783878 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/cf2/599ac29ab5ea472481743dea3e783878 2023-07-24 12:15:08,583 DEBUG [HFileArchiver-1] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/recovered.edits/90.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662/recovered.edits/90.seqid 2023-07-24 12:15:08,585 DEBUG [HFileArchiver-1] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:08,586 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/recovered.edits/90.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169/recovered.edits/90.seqid 2023-07-24 12:15:08,587 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitWithoutPONR/8f319344d8b0142d3e6f98cf87650169 2023-07-24 12:15:08,587 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitWithoutPONR regions 2023-07-24 12:15:08,589 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=40, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:08,592 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitWithoutPONR from hbase:meta 2023-07-24 12:15:08,595 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitWithoutPONR' descriptor. 2023-07-24 12:15:08,596 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=40, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:08,596 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitWithoutPONR' from region states. 2023-07-24 12:15:08,596 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200908596"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:08,596 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200908596"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:08,597 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200908596"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:08,599 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-24 12:15:08,599 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 6ee0f9264c2e2de0329ac96e30326850, NAME => 'testSplitWithoutPONR,,1690200901872.6ee0f9264c2e2de0329ac96e30326850.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 8f319344d8b0142d3e6f98cf87650169, NAME => 'testSplitWithoutPONR,,1690200903901.8f319344d8b0142d3e6f98cf87650169.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 509695ae5946d9f03c670f89496b9662, NAME => 'testSplitWithoutPONR,41,1690200903901.509695ae5946d9f03c670f89496b9662.', STARTKEY => '41', ENDKEY => ''}] 2023-07-24 12:15:08,599 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitWithoutPONR' as deleted. 2023-07-24 12:15:08,599 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitWithoutPONR","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690200908599"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:08,601 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitWithoutPONR state from META 2023-07-24 12:15:08,603 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=40, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:08,605 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR in 82 msec 2023-07-24 12:15:08,626 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=40 2023-07-24 12:15:08,627 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitWithoutPONR, procId: 40 completed 2023-07-24 12:15:08,648 INFO [Listener at localhost/37279] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitWithoutPONR Thread=400 (was 396) Potentially hanging thread: hconnection-0x595be136-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:39462 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/dfs/data/data6/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:44594 [Waiting for operation #16] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-metaLookup-shared--pool-4 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x595be136-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:39326 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:51206 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-metaLookup-shared--pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/dfs/data/data5/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:39444 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/dfs/data/data2/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-6 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:44276 [Waiting for operation #32] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:51456 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-5 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:39482 [Waiting for operation #17] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:51224 [Waiting for operation #23] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:35570 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: HFileArchiver-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:44618 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/dfs/data/data1/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:51280 [Waiting for operation #22] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:51250 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:35582 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:38544 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_305344272_17 at /127.0.0.1:39758 [Waiting for operation #24] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=872 (was 876), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=310 (was 314), ProcessCount=169 (was 169), AvailableMemoryMB=1693 (was 1693) 2023-07-24 12:15:08,663 INFO [Listener at localhost/37279] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=400, OpenFileDescriptor=872, MaxFileDescriptor=60000, SystemLoadAverage=310, ProcessCount=169, AvailableMemoryMB=1693 2023-07-24 12:15:08,664 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:15:08,666 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 12:15:08,667 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(492): Client=jenkins//172.31.2.20 set balanceSwitch=false 2023-07-24 12:15:08,668 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:08,670 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 12:15:08,671 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 12:15:08,673 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:08,674 DEBUG [HFileArchiver-2] backup.HFileArchiver(153): Directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b empty. 2023-07-24 12:15:08,674 DEBUG [HFileArchiver-2] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:08,674 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-07-24 12:15:08,686 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-07-24 12:15:08,688 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => 0030755a278176ce5ddf3114bdcdb65b, NAME => 'testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp 2023-07-24 12:15:08,697 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:08,697 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing 0030755a278176ce5ddf3114bdcdb65b, disabling compactions & flushes 2023-07-24 12:15:08,697 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:08,697 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:08,697 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. after waiting 0 ms 2023-07-24 12:15:08,697 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:08,697 INFO [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:08,697 DEBUG [RegionOpenAndInit-testRollbackAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for 0030755a278176ce5ddf3114bdcdb65b: 2023-07-24 12:15:08,699 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 12:15:08,700 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200908700"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200908700"}]},"ts":"1690200908700"} 2023-07-24 12:15:08,702 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 12:15:08,703 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 12:15:08,703 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200908703"}]},"ts":"1690200908703"} 2023-07-24 12:15:08,704 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLING in hbase:meta 2023-07-24 12:15:08,709 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase6.apache.org=0} racks are {/default-rack=0} 2023-07-24 12:15:08,710 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 12:15:08,710 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 12:15:08,710 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 12:15:08,711 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 12:15:08,711 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN}] 2023-07-24 12:15:08,713 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:15:08,713 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=42, ppid=41, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,41789,1690200894488; forceNewPlan=false, retain=false 2023-07-24 12:15:08,864 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 12:15:08,865 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=0030755a278176ce5ddf3114bdcdb65b, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:08,865 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200908865"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200908865"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200908865"}]},"ts":"1690200908865"} 2023-07-24 12:15:08,867 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=43, ppid=42, state=RUNNABLE; OpenRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488}] 2023-07-24 12:15:09,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:09,026 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0030755a278176ce5ddf3114bdcdb65b, NAME => 'testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:15:09,027 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:09,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:09,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:09,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:09,027 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:09,028 INFO [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:09,030 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1 2023-07-24 12:15:09,030 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1 2023-07-24 12:15:09,030 INFO [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0030755a278176ce5ddf3114bdcdb65b columnFamilyName cf1 2023-07-24 12:15:09,031 INFO [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(310): Store=0030755a278176ce5ddf3114bdcdb65b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:09,031 INFO [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:09,032 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2 2023-07-24 12:15:09,032 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2 2023-07-24 12:15:09,033 INFO [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0030755a278176ce5ddf3114bdcdb65b columnFamilyName cf2 2023-07-24 12:15:09,033 INFO [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(310): Store=0030755a278176ce5ddf3114bdcdb65b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:09,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:09,034 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:09,036 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:15:09,037 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:09,039 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 12:15:09,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 0030755a278176ce5ddf3114bdcdb65b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=71121992, jitterRate=0.05980026721954346}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:15:09,040 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 0030755a278176ce5ddf3114bdcdb65b: 2023-07-24 12:15:09,041 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b., pid=43, masterSystemTime=1690200909019 2023-07-24 12:15:09,042 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:09,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:09,043 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=42 updating hbase:meta row=0030755a278176ce5ddf3114bdcdb65b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,043 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200909043"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200909043"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200909043"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200909043"}]},"ts":"1690200909043"} 2023-07-24 12:15:09,046 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=43, resume processing ppid=42 2023-07-24 12:15:09,047 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=43, ppid=42, state=SUCCESS; OpenRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 in 178 msec 2023-07-24 12:15:09,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=42, resume processing ppid=41 2023-07-24 12:15:09,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=42, ppid=41, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN in 336 msec 2023-07-24 12:15:09,049 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 12:15:09,050 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200909049"}]},"ts":"1690200909049"} 2023-07-24 12:15:09,051 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=ENABLED in hbase:meta 2023-07-24 12:15:09,057 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=41, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 12:15:09,059 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution in 390 msec 2023-07-24 12:15:09,179 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:09,181 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-24 12:15:09,181 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-24 12:15:09,181 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,181 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-24 12:15:09,181 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 12:15:09,182 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,182 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,184 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,184 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,184 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,184 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,184 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,184 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,184 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:09,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,184 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,185 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,185 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,185 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,185 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,185 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,186 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,185 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,186 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,186 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,186 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,186 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,186 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,186 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,186 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,186 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:09,186 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,186 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:09,186 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,187 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,187 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,187 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,187 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,187 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:09,187 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:09,188 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,189 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,189 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,189 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:09,189 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,189 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:09,191 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:09,191 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:09,191 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:09,191 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. started... 2023-07-24 12:15:09,192 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0030755a278176ce5ddf3114bdcdb65b 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 12:15:09,202 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/bbf93197398c4190be29287d5e08cd51 2023-07-24 12:15:09,218 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/d376133b6ec34bb4a5afb4b43a84255f 2023-07-24 12:15:09,222 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/bbf93197398c4190be29287d5e08cd51 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/bbf93197398c4190be29287d5e08cd51 2023-07-24 12:15:09,227 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/bbf93197398c4190be29287d5e08cd51, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:15:09,227 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/d376133b6ec34bb4a5afb4b43a84255f as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/d376133b6ec34bb4a5afb4b43a84255f 2023-07-24 12:15:09,232 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/d376133b6ec34bb4a5afb4b43a84255f, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:15:09,232 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 0030755a278176ce5ddf3114bdcdb65b in 40ms, sequenceid=6, compaction requested=false 2023-07-24 12:15:09,232 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackAndDoubleExecution' 2023-07-24 12:15:09,233 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0030755a278176ce5ddf3114bdcdb65b: 2023-07-24 12:15:09,233 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:09,233 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:09,233 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:09,233 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,233 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:09,233 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:09,235 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,235 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:09,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:09,235 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,236 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:09,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:09,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:09,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,237 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:09,238 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-24 12:15:09,238 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@9de25a3[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:09,238 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 12:15:09,238 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,240 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,240 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,240 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,240 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:09,240 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,240 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,241 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,240 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,241 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:09,240 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:09,241 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:09,241 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:09,241 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:09,241 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:09,241 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:09,241 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:09,241 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:09,242 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:09,242 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,243 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:09,243 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:09,243 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:09,243 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,243 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:09,243 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,243 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,243 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:09,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:09,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:09,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:09,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:09,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,246 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:09,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,247 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,248 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase6.apache.org,41789,1690200894488': 2023-07-24 12:15:09,248 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 12:15:09,248 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-24 12:15:09,249 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:09,249 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:09,249 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-24 12:15:09,249 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:09,250 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,251 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,251 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,250 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:09,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:09,251 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,251 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,251 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:09,251 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,252 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,251 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,252 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,252 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:09,253 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:09,255 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,256 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,256 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,260 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,260 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,260 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,261 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,260 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,261 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,261 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,261 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,261 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,260 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,261 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-24 12:15:09,261 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:09,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,261 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,261 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,262 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:09,261 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,261 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,262 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,262 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:09,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,262 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,262 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,262 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,262 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,262 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,362 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:09,363 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:09,381 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:09,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:09,383 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-24 12:15:09,383 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-24 12:15:09,383 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,383 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-24 12:15:09,383 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 12:15:09,383 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,383 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,385 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,385 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,385 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,385 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,385 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,385 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:09,385 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,385 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,386 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,386 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,386 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,387 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:09,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,387 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,387 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,387 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,387 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:09,387 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,387 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,387 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:09,387 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:09,387 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,387 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,388 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,388 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,388 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,388 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:09,388 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,388 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:09,388 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,388 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,388 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,388 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:09,388 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:09,388 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:09,388 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:09,389 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. started... 2023-07-24 12:15:09,389 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,389 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0030755a278176ce5ddf3114bdcdb65b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:09,392 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,392 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:09,392 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,393 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,393 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:09,403 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/21b67614581447b6abbaff9bffde0ad7 2023-07-24 12:15:09,417 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/8bbf73f57eb84adf9afd4d0424bcd4a1 2023-07-24 12:15:09,422 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/21b67614581447b6abbaff9bffde0ad7 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/21b67614581447b6abbaff9bffde0ad7 2023-07-24 12:15:09,427 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/21b67614581447b6abbaff9bffde0ad7, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:15:09,428 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/8bbf73f57eb84adf9afd4d0424bcd4a1 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/8bbf73f57eb84adf9afd4d0424bcd4a1 2023-07-24 12:15:09,432 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/8bbf73f57eb84adf9afd4d0424bcd4a1, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:15:09,433 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0030755a278176ce5ddf3114bdcdb65b in 44ms, sequenceid=19, compaction requested=false 2023-07-24 12:15:09,433 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0030755a278176ce5ddf3114bdcdb65b: 2023-07-24 12:15:09,433 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:09,433 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:09,433 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:09,433 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,433 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:09,433 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:09,436 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,436 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,436 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:09,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:09,437 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,437 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:09,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:09,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:09,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,439 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:09,439 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-24 12:15:09,439 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@42822d5b[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:09,439 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 12:15:09,439 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,441 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,441 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,441 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,441 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,441 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,441 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:09,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,441 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:09,441 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:09,441 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:09,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,441 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:09,441 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:09,441 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:09,441 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:09,441 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:09,441 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:09,443 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:09,443 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,443 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:09,447 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:09,447 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:09,447 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,447 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,447 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:09,447 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:09,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:09,448 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:09,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:09,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:09,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:09,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,450 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,451 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase6.apache.org,41789,1690200894488': 2023-07-24 12:15:09,451 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 12:15:09,451 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-24 12:15:09,451 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:09,451 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:09,451 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-24 12:15:09,451 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:09,453 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,453 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,453 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,453 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:09,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:09,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,453 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,453 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:09,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,454 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,454 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:09,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,454 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,454 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,455 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,456 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:09,456 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,457 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,459 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,459 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,459 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,459 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,459 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-24 12:15:09,459 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,459 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:09,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,459 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,459 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,459 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,459 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,459 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,460 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:09,460 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,460 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,460 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,460 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,460 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,461 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,461 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,461 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,461 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,461 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,460 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:09,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,561 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:09,562 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:09,577 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:09,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:09,578 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-24 12:15:09,578 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-24 12:15:09,578 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,578 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-24 12:15:09,578 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 12:15:09,579 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,579 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,580 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,580 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,580 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,580 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,580 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,580 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,581 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:09,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,581 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,581 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,581 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,581 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,582 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,582 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,582 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,582 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,582 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:09,582 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,582 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:09,582 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,582 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:09,582 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,582 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,582 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,583 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,582 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,583 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,583 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:09,583 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,583 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:09,583 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:09,583 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:09,583 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,583 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,583 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,583 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:09,583 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:09,583 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:09,584 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. started... 2023-07-24 12:15:09,584 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0030755a278176ce5ddf3114bdcdb65b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:09,585 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,586 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,586 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,586 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:09,586 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,586 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:09,594 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/01a7666e16ed42a394a701ff42dc45e6 2023-07-24 12:15:09,610 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/6890bc547ff04404b822ee57574a3f74 2023-07-24 12:15:09,615 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/01a7666e16ed42a394a701ff42dc45e6 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/01a7666e16ed42a394a701ff42dc45e6 2023-07-24 12:15:09,621 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/01a7666e16ed42a394a701ff42dc45e6, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:15:09,622 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/6890bc547ff04404b822ee57574a3f74 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/6890bc547ff04404b822ee57574a3f74 2023-07-24 12:15:09,626 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/6890bc547ff04404b822ee57574a3f74, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:15:09,627 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0030755a278176ce5ddf3114bdcdb65b in 43ms, sequenceid=32, compaction requested=true 2023-07-24 12:15:09,627 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0030755a278176ce5ddf3114bdcdb65b: 2023-07-24 12:15:09,627 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:09,627 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:09,628 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:09,628 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,628 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:09,628 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:09,629 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,629 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:09,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:09,630 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,630 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:09,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:09,630 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,632 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:09,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-24 12:15:09,632 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@47a52ead[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:09,632 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 12:15:09,632 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,633 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,633 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,633 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,633 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,634 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,634 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,634 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:09,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,634 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:09,634 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:09,634 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:09,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,634 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:09,634 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:09,634 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:09,634 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:09,634 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:09,634 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:09,637 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:09,637 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:09,637 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:09,637 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,637 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,637 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:09,637 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,637 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,637 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,638 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:09,638 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:09,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:09,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase6.apache.org,41789,1690200894488': 2023-07-24 12:15:09,641 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 12:15:09,641 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-24 12:15:09,641 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:09,641 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:09,641 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-24 12:15:09,642 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:09,643 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,643 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,643 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,643 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,643 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,643 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,643 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,644 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:09,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,645 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:09,646 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,646 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,647 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:09,648 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,648 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,648 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,648 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,653 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,653 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,653 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,653 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,653 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,653 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,653 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-24 12:15:09,654 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:09,653 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,654 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,654 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,654 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,654 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,654 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:09,654 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:09,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,654 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,654 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,655 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,655 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,655 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,655 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,655 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,755 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:09,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:09,773 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:09,774 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:09,775 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-24 12:15:09,775 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-24 12:15:09,775 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,775 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-24 12:15:09,775 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 12:15:09,775 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,775 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,778 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,778 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,778 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,778 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,778 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,778 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,778 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,778 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:09,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,779 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,779 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,779 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,779 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,779 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,779 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,780 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,780 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,780 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:09,780 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:09,780 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,780 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,780 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,780 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:09,780 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,780 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,780 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:09,780 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,780 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:09,780 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,780 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,780 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,780 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,780 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,780 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,780 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:09,780 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:09,781 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:09,780 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:09,781 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. started... 2023-07-24 12:15:09,781 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:09,781 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0030755a278176ce5ddf3114bdcdb65b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:09,782 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,783 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,783 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,783 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:09,783 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,783 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:09,791 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/ff3d6ff223ad43e1a578c2e6a61012a8 2023-07-24 12:15:09,805 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/b880b546353140448fa623f389d94ed7 2023-07-24 12:15:09,810 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/ff3d6ff223ad43e1a578c2e6a61012a8 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/ff3d6ff223ad43e1a578c2e6a61012a8 2023-07-24 12:15:09,814 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/ff3d6ff223ad43e1a578c2e6a61012a8, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:15:09,815 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/b880b546353140448fa623f389d94ed7 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/b880b546353140448fa623f389d94ed7 2023-07-24 12:15:09,819 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/b880b546353140448fa623f389d94ed7, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:15:09,820 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0030755a278176ce5ddf3114bdcdb65b in 39ms, sequenceid=45, compaction requested=true 2023-07-24 12:15:09,820 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0030755a278176ce5ddf3114bdcdb65b: 2023-07-24 12:15:09,820 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:09,820 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:09,820 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:09,821 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,821 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:09,821 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:09,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,822 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:09,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:09,823 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,823 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:09,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:09,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:09,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,824 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:09,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-24 12:15:09,825 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@28506b40[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:09,825 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 12:15:09,825 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,827 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,827 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,827 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,827 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:09,828 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,827 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,827 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,828 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,828 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:09,828 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:09,828 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:09,828 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:09,828 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:09,828 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:09,828 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:09,828 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:09,828 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:09,829 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:09,829 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,829 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,829 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:09,829 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:09,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:09,830 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:09,830 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,830 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:09,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:09,830 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:09,830 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,830 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:09,830 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:09,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,831 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:09,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,832 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,833 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase6.apache.org,41789,1690200894488': 2023-07-24 12:15:09,833 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 12:15:09,833 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-24 12:15:09,833 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:09,833 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:09,833 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-24 12:15:09,833 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:09,835 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,835 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,835 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,835 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,835 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,835 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:09,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:09,835 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,835 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,835 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:09,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,836 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,836 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,836 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,837 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:09,837 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,837 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,838 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:09,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:09,839 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,839 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,839 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,840 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,843 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,843 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,843 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,843 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,843 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,843 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:09,843 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-24 12:15:09,843 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:09,843 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,843 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:09,843 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,844 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:09,844 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:09,843 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:09,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,844 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:09,844 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,844 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:09,845 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,845 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,845 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:09,845 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,845 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,944 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:09,945 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:09,961 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:09,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:09,963 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-24 12:15:09,963 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-24 12:15:09,963 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,963 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-24 12:15:09,963 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 12:15:09,964 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,964 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,965 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,965 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,965 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,965 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:09,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:09,965 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:09,965 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:09,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:09,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,966 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:09,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,967 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,967 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:09,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:09,967 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:09,967 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:09,967 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:09,967 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,967 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,967 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,967 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,967 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:09,967 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,967 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:09,967 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,967 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,967 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:09,968 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:09,967 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:09,968 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,968 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:09,968 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:09,968 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:09,968 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:09,969 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,970 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,970 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,970 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:09,970 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:09,970 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:09,971 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:09,971 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:09,971 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:09,971 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. started... 2023-07-24 12:15:09,971 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0030755a278176ce5ddf3114bdcdb65b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:09,986 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/69bebdfeb3144f59a9b55ee23b35a9e1 2023-07-24 12:15:09,999 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/456c894be60f41bf843e260fc392b6dc 2023-07-24 12:15:10,004 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/69bebdfeb3144f59a9b55ee23b35a9e1 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/69bebdfeb3144f59a9b55ee23b35a9e1 2023-07-24 12:15:10,009 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/69bebdfeb3144f59a9b55ee23b35a9e1, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:15:10,010 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/456c894be60f41bf843e260fc392b6dc as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/456c894be60f41bf843e260fc392b6dc 2023-07-24 12:15:10,016 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/456c894be60f41bf843e260fc392b6dc, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:15:10,017 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0030755a278176ce5ddf3114bdcdb65b in 45ms, sequenceid=58, compaction requested=true 2023-07-24 12:15:10,017 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0030755a278176ce5ddf3114bdcdb65b: 2023-07-24 12:15:10,017 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:10,017 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:10,017 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:10,017 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,017 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:10,017 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:10,019 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,020 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:10,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:10,020 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:10,020 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:10,020 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:10,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:10,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,021 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,022 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:10,022 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-24 12:15:10,022 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@50dba0d4[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:10,022 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 12:15:10,022 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,024 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,024 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,024 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,024 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,024 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,024 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:10,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,024 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:10,024 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:10,024 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:10,024 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:10,024 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:10,024 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,024 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:10,024 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:10,024 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:10,024 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:10,026 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:10,026 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:10,026 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:10,030 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,030 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:10,030 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:10,030 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:10,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:10,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:10,030 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:10,030 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:10,030 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:10,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:10,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:10,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:10,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:10,032 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:10,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,034 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase6.apache.org,41789,1690200894488': 2023-07-24 12:15:10,034 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 12:15:10,034 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-24 12:15:10,034 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:10,034 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:10,034 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-24 12:15:10,034 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:10,035 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,035 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:10,036 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,036 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:10,036 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:10,035 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,036 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:10,036 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:10,036 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:10,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:10,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:10,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,037 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:10,037 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:10,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:10,037 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:10,039 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,039 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,040 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,042 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,043 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:10,043 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:10,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:10,043 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:10,043 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:10,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-24 12:15:10,043 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:10,042 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:10,043 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,043 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:10,043 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:10,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:10,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:10,043 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:10,043 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:10,043 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:10,043 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:10,043 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:10,043 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:10,043 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:10,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:10,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:10,044 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:10,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:10,044 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,044 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,044 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,044 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,044 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,044 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:10,044 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:10,144 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:10,145 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:10,159 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:10,160 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testRollbackAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:10,161 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testRollbackAndDoubleExecution 2023-07-24 12:15:10,161 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRollbackAndDoubleExecution' 2023-07-24 12:15:10,161 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:10,161 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRollbackAndDoubleExecution' starting 'acquire' 2023-07-24 12:15:10,161 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRollbackAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 12:15:10,161 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,161 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:10,162 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:10,162 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:10,162 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:10,162 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:10,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:10,162 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:10,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:10,163 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:10,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:10,163 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,163 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:10,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:10,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:10,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:10,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,164 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:10,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:10,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:10,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:10,164 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:10,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:10,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:10,164 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:10,164 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRollbackAndDoubleExecution 2023-07-24 12:15:10,164 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:10,164 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:10,165 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:10,165 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRollbackAndDoubleExecution is still present, removing 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:10,165 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:10,165 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRollbackAndDoubleExecution 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRollbackAndDoubleExecution' with timeout 60000ms 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRollbackAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:10,165 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:10,165 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:10,165 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. started... 2023-07-24 12:15:10,165 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 0030755a278176ce5ddf3114bdcdb65b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:10,168 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,169 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,169 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,169 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:10,169 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,169 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:10,176 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/75a74fb05121409396d7ae1c3c6310bf 2023-07-24 12:15:10,189 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/77153bf8b3564490b09ec4f07cf58109 2023-07-24 12:15:10,194 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/75a74fb05121409396d7ae1c3c6310bf as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/75a74fb05121409396d7ae1c3c6310bf 2023-07-24 12:15:10,199 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/75a74fb05121409396d7ae1c3c6310bf, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:15:10,200 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/77153bf8b3564490b09ec4f07cf58109 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/77153bf8b3564490b09ec4f07cf58109 2023-07-24 12:15:10,205 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/77153bf8b3564490b09ec4f07cf58109, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:15:10,206 INFO [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 0030755a278176ce5ddf3114bdcdb65b in 41ms, sequenceid=71, compaction requested=true 2023-07-24 12:15:10,206 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 0030755a278176ce5ddf3114bdcdb65b: 2023-07-24 12:15:10,206 DEBUG [rs(jenkins-hbase6.apache.org,41789,1690200894488)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:10,206 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:10,206 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:10,206 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,206 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRollbackAndDoubleExecution' locally acquired 2023-07-24 12:15:10,207 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testRollbackAndDoubleExecution) in zk 2023-07-24 12:15:10,208 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,208 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,208 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:10,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:10,209 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,209 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRollbackAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:10,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:10,209 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:10,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:10,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:10,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure 'testRollbackAndDoubleExecution' on coordinator 2023-07-24 12:15:10,211 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@3e0b761[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:10,211 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRollbackAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 12:15:10,211 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,212 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,212 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,212 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,213 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,213 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:10,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,213 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:10,213 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,213 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:10,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,213 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,213 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:10,213 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRollbackAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:15:10,213 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:10,213 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:10,213 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:10,213 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRollbackAndDoubleExecution' locally completed 2023-07-24 12:15:10,213 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRollbackAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:10,216 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:10,216 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:10,216 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:10,217 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,217 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:10,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:10,217 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:10,217 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:10,217 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:10,218 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRollbackAndDoubleExecution' has notified controller of completion 2023-07-24 12:15:10,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:10,218 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:10,218 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRollbackAndDoubleExecution' completed. 2023-07-24 12:15:10,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:10,218 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:10,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:10,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:10,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRollbackAndDoubleExecution' member 'jenkins-hbase6.apache.org,41789,1690200894488': 2023-07-24 12:15:10,221 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' released barrier for procedure'testRollbackAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 12:15:10,221 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRollbackAndDoubleExecution' execution completed 2023-07-24 12:15:10,221 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:10,221 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:10,221 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRollbackAndDoubleExecution 2023-07-24 12:15:10,221 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRollbackAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:10,223 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,223 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:10,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:10,223 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,223 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:10,223 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,223 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:10,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:10,223 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,223 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:10,223 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,224 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:10,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:10,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:10,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:10,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:10,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:10,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:10,225 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:10,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,226 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:10,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:10,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRollbackAndDoubleExecution 2023-07-24 12:15:10,227 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,228 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,228 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,234 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:10,234 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:10,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:10,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:10,234 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:10,234 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:10,234 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,234 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRollbackAndDoubleExecution' 2023-07-24 12:15:10,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:10,234 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:10,234 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:10,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:10,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:10,234 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:10,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:10,234 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:10,234 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:10,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:10,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:10,235 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,235 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,235 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRollbackAndDoubleExecution 2023-07-24 12:15:10,235 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:10,235 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRollbackAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:10,235 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:10,235 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,235 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:10,235 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:10,235 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:10,235 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:10,235 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRollbackAndDoubleExecution 2023-07-24 12:15:10,236 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRollbackAndDoubleExecution 2023-07-24 12:15:10,236 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:10,336 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:10,337 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:10,598 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-07-24 12:15:10,599 DEBUG [Listener at localhost/37279] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:10,600 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:10,601 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:10,602 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:10,602 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:10,602 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:10,602 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:10,602 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:10,603 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:10,850 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 2 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:10,851 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:10,851 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:10,851 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:10,851 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-7,5,PEWorkerGroup] Thread[HFileArchiver-8,5,PEWorkerGroup] 2023-07-24 12:15:10,851 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:10,851 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:10,851 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:10,851 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:10,852 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:10,854 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-07-24 12:15:10,854 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:10,854 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:10,855 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:10,855 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:10,855 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:10,855 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:10,856 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:10,856 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:10,856 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:10,856 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:10,856 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:10,857 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:10,857 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:10,857 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:10,857 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:10,857 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:10,857 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:10,857 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:10,858 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 12:15:10,858 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:10,859 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:10,859 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200940859 2023-07-24 12:15:10,861 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:10,861 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:10,864 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:10,864 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:10,864 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:10,864 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200970864 2023-07-24 12:15:10,864 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201030864 2023-07-24 12:15:10,864 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:10,865 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:10,866 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:10,867 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:10,867 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:10,867 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:11,115 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 3 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,115 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:11,115 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:11,115 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:11,115 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:11,115 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:11,115 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:11,115 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:11,116 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:11,116 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:11,118 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-24 12:15:11,118 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:11,119 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:11,119 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:11,119 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:11,120 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:11,120 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:11,121 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:11,121 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:11,121 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:11,122 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:11,122 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:11,122 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:11,122 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:11,123 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,123 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:11,123 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,123 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,123 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,123 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-24 12:15:11,124 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:11,124 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:11,124 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200941124 2023-07-24 12:15:11,126 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,126 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:11,129 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:11,129 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:11,130 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:11,130 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200971130 2023-07-24 12:15:11,130 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201031130 2023-07-24 12:15:11,130 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:11,130 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:11,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN}] 2023-07-24 12:15:11,133 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:11,133 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,133 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:11,133 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:11,380 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,380 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:11,380 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:11,380 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:11,381 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:11,381 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:11,381 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:11,381 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:11,381 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:11,382 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:11,383 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-24 12:15:11,384 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:11,384 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:11,385 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:11,385 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:11,385 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:11,386 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:11,386 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:11,386 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:11,387 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:11,387 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:11,388 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:11,388 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:11,389 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:11,389 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,389 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:11,389 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,389 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,389 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,390 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-24 12:15:11,390 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:11,391 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:11,391 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200941391 2023-07-24 12:15:11,392 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:11,393 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,395 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:11,395 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:11,395 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:11,396 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200971396 2023-07-24 12:15:11,396 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201031396 2023-07-24 12:15:11,396 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:11,396 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN}] 2023-07-24 12:15:11,396 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:11,398 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:11,399 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:11,399 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200911399"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200911399"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200911399"}]},"ts":"1690200911399"} 2023-07-24 12:15:11,401 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488}] 2023-07-24 12:15:11,401 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:11,401 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:11,401 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:11,401 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:11,646 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,646 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:11,646 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:11,646 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:11,646 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:11,647 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:11,647 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:11,647 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:11,647 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:11,648 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:11,649 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 12:15:11,649 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-24 12:15:11,649 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:11,649 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:11,649 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:11,650 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:11,650 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:11,650 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:11,650 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:11,650 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:11,651 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:11,651 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:11,651 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:11,651 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:11,652 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:11,652 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,652 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:11,652 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:11,653 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,653 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,653 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,653 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:11,653 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:11,653 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 12:15:11,653 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:11,654 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:11,654 INFO [Listener at localhost/37279] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b to restore RIT 2023-07-24 12:15:11,654 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200941654 2023-07-24 12:15:11,655 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:11,655 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:11,657 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:11,658 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSING, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:11,658 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:11,658 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200971658 2023-07-24 12:15:11,658 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201031658 2023-07-24 12:15:11,658 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 12:15:11,658 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:11,658 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200911658"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200911658"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200911658"}]},"ts":"1690200911658"} 2023-07-24 12:15:11,660 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488}] 2023-07-24 12:15:11,660 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:11,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:11,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 0030755a278176ce5ddf3114bdcdb65b, disabling compactions & flushes 2023-07-24 12:15:11,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:11,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:11,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. after waiting 0 ms 2023-07-24 12:15:11,813 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:11,813 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2745): Flushing 0030755a278176ce5ddf3114bdcdb65b 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-24 12:15:11,826 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/9b6acb4cd1ca40a2916f677d43ee62e2 2023-07-24 12:15:11,840 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/026e0e33452a47efa5838eee95abb353 2023-07-24 12:15:11,845 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/9b6acb4cd1ca40a2916f677d43ee62e2 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/9b6acb4cd1ca40a2916f677d43ee62e2 2023-07-24 12:15:11,849 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/9b6acb4cd1ca40a2916f677d43ee62e2, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 12:15:11,850 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/026e0e33452a47efa5838eee95abb353 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/026e0e33452a47efa5838eee95abb353 2023-07-24 12:15:11,854 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/026e0e33452a47efa5838eee95abb353, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 12:15:11,855 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 0030755a278176ce5ddf3114bdcdb65b in 42ms, sequenceid=82, compaction requested=true 2023-07-24 12:15:11,862 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-24 12:15:11,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:11,863 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:11,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 0030755a278176ce5ddf3114bdcdb65b: 2023-07-24 12:15:11,865 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:11,866 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED 2023-07-24 12:15:11,866 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200911866"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200911866"}]},"ts":"1690200911866"} 2023-07-24 12:15:11,867 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:11,867 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:11,867 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:11,867 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=46, ppid=45, state=SUCCESS, locked=true; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:11,908 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,908 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:11,908 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:11,908 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:11,910 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:11,911 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:11,911 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:11,911 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:11,911 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:11,912 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:11,912 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 12:15:11,913 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-24 12:15:11,913 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4] 2023-07-24 12:15:11,913 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:11,913 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:11,913 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:11,913 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:11,914 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:11,914 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:11,914 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:11,915 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:11,915 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:11,915 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:11,916 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:11,916 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:11,916 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:11,917 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,917 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:11,917 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:11,917 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:11,917 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,918 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,918 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:11,918 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:11,918 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:11,918 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:11,918 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-24 12:15:11,919 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:11,919 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:11,920 INFO [Listener at localhost/37279] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b to restore RIT 2023-07-24 12:15:11,923 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200941923 2023-07-24 12:15:11,924 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=46, ppid=45, state=RUNNABLE; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:11,924 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:11,926 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:11,927 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:11,927 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:11,927 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200971927 2023-07-24 12:15:11,927 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201031927 2023-07-24 12:15:11,927 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:11,927 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=45 updating hbase:meta row=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:11,928 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200911927"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200911927"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200911927"}]},"ts":"1690200911927"} 2023-07-24 12:15:11,929 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:11,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=46, resume processing ppid=45 2023-07-24 12:15:11,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 in 269 msec 2023-07-24 12:15:11,932 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:11,932 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:11,932 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:11,932 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=45, ppid=44, state=SUCCESS, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:12,177 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 11 exec state=pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:12,178 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:12,178 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:12,178 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:12,179 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:12,179 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:12,179 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:12,179 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:12,179 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:12,180 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:12,181 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 12:15:12,181 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3] 2023-07-24 12:15:12,181 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:15:12,182 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:12,182 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:12,182 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:12,182 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:12,182 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:12,183 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:12,183 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:12,183 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:12,183 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:12,184 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:12,184 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:12,184 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:12,184 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:12,185 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:12,185 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:12,185 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:12,185 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:12,185 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:12,186 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:12,186 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:12,186 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:12,186 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:12,186 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 12:15:12,186 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:12,187 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:12,187 INFO [Listener at localhost/37279] assignment.AssignmentManager(305): Attach pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b to restore RIT 2023-07-24 12:15:12,187 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200942187 2023-07-24 12:15:12,188 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:12,188 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:12,190 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:12,190 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:12,190 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:12,190 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200972190 2023-07-24 12:15:12,191 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201032191 2023-07-24 12:15:12,191 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 12:15:12,191 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=45, ppid=44, state=RUNNABLE:REGION_STATE_TRANSITION_CONFIRM_CLOSED; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:12,192 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:12,195 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=45, resume processing ppid=44 2023-07-24 12:15:12,195 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN in 796 msec 2023-07-24 12:15:12,198 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:12,198 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:12,198 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:12,198 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:12,441 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 4 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:12,441 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:12,441 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:12,441 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:12,441 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:12,441 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:12,441 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:12,442 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:12,442 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:12,442 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:12,443 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 12:15:12,443 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:15:12,444 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:15:12,444 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:12,444 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:12,445 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:12,445 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:12,445 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:12,446 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:12,446 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:12,446 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:12,446 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:12,447 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:12,447 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:12,447 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:12,448 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:12,448 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:12,448 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:12,449 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:12,449 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:12,449 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:12,449 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:12,449 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-24 12:15:12,449 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:12,450 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:12,450 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200942450 2023-07-24 12:15:12,452 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:12,452 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:12,454 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:12,455 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:12,455 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:12,455 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200972455 2023-07-24 12:15:12,455 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201032455 2023-07-24 12:15:12,455 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:12,455 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:12,455 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:12,455 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:12,461 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:12,472 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=0030755a278176ce5ddf3114bdcdb65b, threads=14 2023-07-24 12:15:12,473 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/026e0e33452a47efa5838eee95abb353 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,473 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/456c894be60f41bf843e260fc392b6dc for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,473 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/6890bc547ff04404b822ee57574a3f74 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,475 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/77153bf8b3564490b09ec4f07cf58109 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,479 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/8bbf73f57eb84adf9afd4d0424bcd4a1 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,479 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/b880b546353140448fa623f389d94ed7 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,480 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/d376133b6ec34bb4a5afb4b43a84255f for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,480 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/21b67614581447b6abbaff9bffde0ad7 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,480 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/01a7666e16ed42a394a701ff42dc45e6 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,481 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/69bebdfeb3144f59a9b55ee23b35a9e1 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,481 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/75a74fb05121409396d7ae1c3c6310bf for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,483 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/9b6acb4cd1ca40a2916f677d43ee62e2 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,487 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/bbf93197398c4190be29287d5e08cd51 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,487 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/ff3d6ff223ad43e1a578c2e6a61012a8 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,667 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/026e0e33452a47efa5838eee95abb353 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,696 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/77153bf8b3564490b09ec4f07cf58109 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,698 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/b880b546353140448fa623f389d94ed7 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,701 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/8bbf73f57eb84adf9afd4d0424bcd4a1 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,706 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/bbf93197398c4190be29287d5e08cd51 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,719 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/01a7666e16ed42a394a701ff42dc45e6 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,723 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/d376133b6ec34bb4a5afb4b43a84255f for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,743 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/ff3d6ff223ad43e1a578c2e6a61012a8 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,747 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/456c894be60f41bf843e260fc392b6dc for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,751 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/21b67614581447b6abbaff9bffde0ad7 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,752 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/69bebdfeb3144f59a9b55ee23b35a9e1 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,752 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/9b6acb4cd1ca40a2916f677d43ee62e2 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,754 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/75a74fb05121409396d7ae1c3c6310bf for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,755 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/6890bc547ff04404b822ee57574a3f74 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:12,755 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 0030755a278176ce5ddf3114bdcdb65b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 12:15:13,135 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 12:15:13,217 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:13,217 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,217 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:13,217 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:13,455 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 5 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,456 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:13,456 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:13,456 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:13,456 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:13,456 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:13,456 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:13,456 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:13,456 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:13,457 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:13,458 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-24 12:15:13,458 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:15:13,458 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:15:13,458 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:13,459 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:13,459 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:13,459 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:13,460 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:13,460 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:13,460 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:13,460 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:13,461 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:13,461 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:13,461 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:13,461 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:13,461 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:13,462 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,462 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:13,462 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:13,462 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:13,462 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,462 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,462 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,462 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:13,462 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:13,462 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 12:15:13,463 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:13,464 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:13,464 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200943464 2023-07-24 12:15:13,465 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:13,466 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,468 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:13,468 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:13,468 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:13,468 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200973468 2023-07-24 12:15:13,468 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201033468 2023-07-24 12:15:13,468 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:13,477 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=44 splitting 14 storefiles, region=0030755a278176ce5ddf3114bdcdb65b, threads=14 2023-07-24 12:15:13,478 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/026e0e33452a47efa5838eee95abb353 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,478 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/456c894be60f41bf843e260fc392b6dc for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,478 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/6890bc547ff04404b822ee57574a3f74 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,478 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/77153bf8b3564490b09ec4f07cf58109 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,478 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/8bbf73f57eb84adf9afd4d0424bcd4a1 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,478 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/b880b546353140448fa623f389d94ed7 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,479 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/d376133b6ec34bb4a5afb4b43a84255f for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,479 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/01a7666e16ed42a394a701ff42dc45e6 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,480 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/21b67614581447b6abbaff9bffde0ad7 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,480 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/69bebdfeb3144f59a9b55ee23b35a9e1 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,481 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/75a74fb05121409396d7ae1c3c6310bf for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,481 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/9b6acb4cd1ca40a2916f677d43ee62e2 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,481 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/bbf93197398c4190be29287d5e08cd51 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,481 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=44 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/ff3d6ff223ad43e1a578c2e6a61012a8 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,667 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/77153bf8b3564490b09ec4f07cf58109 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,675 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/8bbf73f57eb84adf9afd4d0424bcd4a1 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,686 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/6890bc547ff04404b822ee57574a3f74 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,688 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/d376133b6ec34bb4a5afb4b43a84255f for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,703 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/ff3d6ff223ad43e1a578c2e6a61012a8 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,717 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/b880b546353140448fa623f389d94ed7 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,717 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/456c894be60f41bf843e260fc392b6dc for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,735 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/69bebdfeb3144f59a9b55ee23b35a9e1 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,736 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/9b6acb4cd1ca40a2916f677d43ee62e2 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,738 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/01a7666e16ed42a394a701ff42dc45e6 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,739 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/026e0e33452a47efa5838eee95abb353 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,739 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/bbf93197398c4190be29287d5e08cd51 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,747 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/75a74fb05121409396d7ae1c3c6310bf for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,758 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=44 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/21b67614581447b6abbaff9bffde0ad7 for region: 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:13,758 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=44 split storefiles for region 0030755a278176ce5ddf3114bdcdb65b Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 12:15:13,809 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:13,816 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/9d2e4ba64e3372b74270d6f2ef4c0490/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 12:15:13,818 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/054dd5a119895a40f5f57d00e79d7def/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 12:15:13,818 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:13,818 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,818 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:13,819 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:13,969 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(417): Restart 6 exec state=pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,969 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:13,969 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:13,969 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:13,969 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:13,969 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:13,969 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:13,969 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:13,969 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:13,970 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:13,971 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-24 12:15:13,971 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:15:13,971 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:15:13,972 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:13,972 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:13,972 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:13,972 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:13,973 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:13,973 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:13,973 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:13,973 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:13,973 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:13,974 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:13,974 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:13,974 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:13,974 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:13,975 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,975 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:13,975 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:13,977 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:13,977 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,977 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,977 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,977 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:13,978 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:13,978 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-24 12:15:13,978 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:13,979 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:13,980 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200943979 2023-07-24 12:15:13,980 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:13,981 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,983 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:13,983 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:13,983 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:13,983 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200973983 2023-07-24 12:15:13,983 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201033983 2023-07-24 12:15:13,983 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:13,986 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:13,989 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:13,989 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:13,989 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:13,989 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:14,234 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(509): Restart 0 rollback state: pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,234 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:14,234 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:14,234 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:14,235 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:14,235 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:14,235 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:14,235 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:14,235 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:14,236 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:14,237 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-24 12:15:14,237 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:15:14,237 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:15:14,237 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:14,237 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:14,238 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:14,238 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:14,238 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:14,238 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:14,238 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:14,238 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:14,239 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:14,239 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:14,239 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:14,240 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:14,240 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:14,240 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,241 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:14,241 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:14,241 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:14,241 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,241 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,241 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,241 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:14,241 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:14,242 DEBUG [Listener at localhost/37279] procedure2.StateMachineProcedure(225): Abort requested for pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,242 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 12:15:14,242 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:14,243 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:14,243 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200944243 2023-07-24 12:15:14,246 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=44, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,246 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:14,248 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:14,249 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:14,249 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:14,249 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200974249 2023-07-24 12:15:14,249 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201034249 2023-07-24 12:15:14,249 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 12:15:14,250 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:14,263 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:14,263 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-24 12:15:14,263 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:14,263 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:14,500 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(509): Restart 1 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,501 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:14,501 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:14,501 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:14,501 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:14,501 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:14,501 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:14,501 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:14,501 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:14,502 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:14,503 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-07-24 12:15:14,503 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:15:14,503 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:15:14,503 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:14,504 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:14,504 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:14,504 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:14,504 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:14,504 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:14,504 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:14,505 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:14,505 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:14,505 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:14,505 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:14,505 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:14,506 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:14,507 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,507 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:14,507 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:14,507 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:14,507 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,507 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,507 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,507 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:14,507 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:14,507 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 12:15:14,508 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:14,508 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:14,509 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200944509 2023-07-24 12:15:14,511 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:14,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,513 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:14,514 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:14,514 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:14,514 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200974514 2023-07-24 12:15:14,514 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201034514 2023-07-24 12:15:14,514 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 12:15:14,514 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:14,516 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:14,516 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-24 12:15:14,516 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:14,516 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:14,765 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(509): Restart 2 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,765 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:14,765 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:14,765 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:14,766 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:14,766 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:14,766 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:14,766 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:14,766 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:14,767 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:14,768 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-24 12:15:14,768 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:15:14,768 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:15:14,768 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:14,769 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:14,769 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:14,769 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:14,769 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:14,769 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:14,770 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:14,770 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:14,770 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:14,770 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:14,770 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:14,770 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:14,771 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:14,771 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,772 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:14,772 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:14,772 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:14,772 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,772 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,772 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,772 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:14,772 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:14,772 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 12:15:14,773 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:14,773 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:14,774 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200944774 2023-07-24 12:15:14,774 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:14,775 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:14,777 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:14,777 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:14,777 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:14,777 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200974777 2023-07-24 12:15:14,778 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201034778 2023-07-24 12:15:14,778 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:14,778 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:14,780 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:14,780 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-24 12:15:14,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:14,781 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:15,028 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(509): Restart 3 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,028 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:15,028 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:15,028 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:15,029 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:15,029 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:15,029 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:15,029 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:15,029 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:15,030 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:15,031 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-24 12:15:15,031 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:15:15,031 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:15:15,031 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:15,032 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:15,032 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:15,032 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:15,032 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:15,032 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:15,032 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:15,033 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:15,033 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:15,033 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:15,033 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:15,033 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:15,033 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:15,034 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,034 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:15,034 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:15,034 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:15,034 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,034 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,034 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,034 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:15,034 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:15,035 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 12:15:15,035 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:15,036 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:15,036 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200945036 2023-07-24 12:15:15,037 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:15,038 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,040 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:15,040 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:15,040 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:15,040 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200975040 2023-07-24 12:15:15,040 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201035040 2023-07-24 12:15:15,040 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:15,040 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:15,044 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1054): Stored [pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN] 2023-07-24 12:15:15,044 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:15,044 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-24 12:15:15,044 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:15,044 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:15,290 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(509): Restart 4 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,290 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:15,290 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:15,291 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:15,291 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:15,291 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:15,291 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:15,291 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:15,291 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:15,292 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:15,293 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-24 12:15:15,293 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=45, ppid=44, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:15:15,293 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=46, ppid=45, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:15:15,293 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-24 12:15:15,293 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:15,293 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:15,294 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:15,294 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:15,294 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:15,294 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:15,294 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:15,294 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:15,295 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:15,295 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:15,295 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:15,295 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:15,295 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:15,296 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,296 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:15,296 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:15,296 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:15:15,297 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:15,297 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,297 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,297 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,297 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=45, ppid=44, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:15,297 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=46, ppid=45, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:15,297 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:15,297 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 12:15:15,298 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:15,298 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:15,298 INFO [Listener at localhost/37279] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b to restore RIT 2023-07-24 12:15:15,298 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200945298 2023-07-24 12:15:15,300 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:15,300 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,301 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:15:15,304 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:15,304 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:15,304 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:15,304 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200975304 2023-07-24 12:15:15,304 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201035304 2023-07-24 12:15:15,304 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:15,304 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:15,309 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:15,309 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-24 12:15:15,309 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:15,309 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:15,554 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(509): Restart 5 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,554 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:15,554 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:15,554 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:15,555 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:15,555 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:15,555 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:15,555 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:15,555 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:15,556 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:15,556 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 12:15:15,557 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-24 12:15:15,557 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:15,557 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:15,557 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:15,557 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:15,557 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:15,558 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:15,558 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:15,558 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:15,558 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:15,558 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:15,559 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:15,559 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:15,559 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:15,559 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,559 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:15:15,559 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:15,559 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,560 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,560 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,560 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:15,560 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-07-24 12:15:15,560 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:15,561 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:15,561 INFO [Listener at localhost/37279] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b to restore RIT 2023-07-24 12:15:15,561 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200945561 2023-07-24 12:15:15,562 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:15,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,564 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:15:15,566 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:15,567 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:15,567 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:15,567 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200975567 2023-07-24 12:15:15,567 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201035567 2023-07-24 12:15:15,567 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 12:15:15,567 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:15,569 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:15,569 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-24 12:15:15,570 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:15,570 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:15,817 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(509): Restart 6 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,817 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:15,817 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:15,817 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:15,818 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:15,818 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:15,818 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:15,818 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:15,818 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:15,819 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:15,819 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-24 12:15:15,819 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-24 12:15:15,820 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:15,820 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:15,820 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:15,820 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:15,821 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:15,821 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:15,821 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:15,821 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:15,821 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:15,822 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:15,822 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:15,822 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:15,822 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:15,822 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,822 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:15:15,822 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:15,823 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,823 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,823 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,823 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:15,823 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 4 msec 2023-07-24 12:15:15,823 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:15,824 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:15,824 INFO [Listener at localhost/37279] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b to restore RIT 2023-07-24 12:15:15,824 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200945824 2023-07-24 12:15:15,826 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:15,827 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PRE_OPERATION, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:15,828 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:15:15,830 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:15,831 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:15,831 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:15,831 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200975831 2023-07-24 12:15:15,831 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201035831 2023-07-24 12:15:15,831 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-07-24 12:15:15,831 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:15,834 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:15,834 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1581): TESTING: Kill before store update 2023-07-24 12:15:15,834 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:15,834 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill before store update at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1583) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeRollback(ProcedureExecutor.java:1498) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1349) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:16,081 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(509): Restart 7 rollback state: pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, locked=true, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:16,081 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:16,081 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:16,082 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:16,082 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:16,082 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:16,082 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:16,082 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:16,082 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:16,083 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:16,083 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=44, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-24 12:15:16,084 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[] 2023-07-24 12:15:16,084 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:16,084 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:16,084 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:16,084 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:16,085 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:16,085 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:16,085 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:16,085 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:16,085 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:16,086 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:16,086 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:16,086 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:16,087 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:16,087 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:16,087 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:15:16,087 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:16,087 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:16,088 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:16,088 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:16,088 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:15:16,088 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 5 msec 2023-07-24 12:15:16,089 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:16,089 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:16,089 INFO [Listener at localhost/37279] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b to restore RIT 2023-07-24 12:15:16,090 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200946090 2023-07-24 12:15:16,090 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:16,091 INFO [PEWorker-1] procedure2.ProcedureExecutor(1357): LOCK_EVENT_WAIT rollback...pid=44, state=FAILED:SPLIT_TABLE_REGION_PREPARE, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:16,095 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:15:16,096 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:16,096 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:16,097 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:16,097 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200976097 2023-07-24 12:15:16,097 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201036097 2023-07-24 12:15:16,097 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 6 msec 2023-07-24 12:15:16,097 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:15:16,099 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def exec-time=5.4980 sec 2023-07-24 12:15:16,099 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:15:16,100 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN; state=CLOSED, location=jenkins-hbase6.apache.org,41789,1690200894488; forceNewPlan=false, retain=true 2023-07-24 12:15:16,250 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 12:15:16,251 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=0030755a278176ce5ddf3114bdcdb65b, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:16,251 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200916251"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200916251"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200916251"}]},"ts":"1690200916251"} 2023-07-24 12:15:16,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488}] 2023-07-24 12:15:16,253 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:15:16,253 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:15:16,253 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:15:16,254 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=47, state=WAITING:REGION_STATE_TRANSITION_CONFIRM_OPENED, locked=true; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:15:16,347 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:15:16,347 WARN [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(526): ProcedureExecutor not running, may have been stopped by pending procedure due to KillAndToggleBeforeStoreUpdate flag. 2023-07-24 12:15:16,347 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:15:16,347 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:15:16,347 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:15:16,347 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:15:16,348 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:15:16,348 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:15:16,348 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:15:16,348 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:15:16,348 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:15:16,349 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=47, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[0] 2023-07-24 12:15:16,350 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:15:16,350 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:15:16,350 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:15:16,350 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:15:16,350 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:16,350 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:15:16,351 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:16,351 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:15:16,351 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:16,351 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:15:16,351 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:16,351 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:15:16,352 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:16,352 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:15:16,352 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:15:16,352 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:15:16,352 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:15:16,352 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 3 msec 2023-07-24 12:15:16,353 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:15:16,353 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:15:16,353 INFO [Listener at localhost/37279] assignment.AssignmentManager(305): Attach pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN to state=OFFLINE, location=null, table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b to restore RIT 2023-07-24 12:15:16,358 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690200946358 2023-07-24 12:15:16,358 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:15:16,360 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=47, state=RUNNABLE:REGION_STATE_TRANSITION_OPEN; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:15:16,362 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:15:16,362 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=0030755a278176ce5ddf3114bdcdb65b, regionState=OPENING, lastHost=jenkins-hbase6.apache.org,41789,1690200894488, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488, openSeqNum=2 2023-07-24 12:15:16,362 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:15:16,362 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690200976362 2023-07-24 12:15:16,362 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201036362 2023-07-24 12:15:16,362 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:15:16,362 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=0030755a278176ce5ddf3114bdcdb65b, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:16,363 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200916362"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200916362"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200916362"}]},"ts":"1690200916362"} 2023-07-24 12:15:16,364 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=48, ppid=47, state=RUNNABLE; OpenRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488}] 2023-07-24 12:15:16,519 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:16,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 0030755a278176ce5ddf3114bdcdb65b, NAME => 'testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:15:16,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:16,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackAndDoubleExecution 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:16,520 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:16,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:16,521 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:16,522 INFO [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:16,523 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1 2023-07-24 12:15:16,523 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1 2023-07-24 12:15:16,523 INFO [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0030755a278176ce5ddf3114bdcdb65b columnFamilyName cf1 2023-07-24 12:15:16,530 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/01a7666e16ed42a394a701ff42dc45e6 2023-07-24 12:15:16,534 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/21b67614581447b6abbaff9bffde0ad7 2023-07-24 12:15:16,537 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/69bebdfeb3144f59a9b55ee23b35a9e1 2023-07-24 12:15:16,540 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/75a74fb05121409396d7ae1c3c6310bf 2023-07-24 12:15:16,544 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/9b6acb4cd1ca40a2916f677d43ee62e2 2023-07-24 12:15:16,547 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/bbf93197398c4190be29287d5e08cd51 2023-07-24 12:15:16,550 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/ff3d6ff223ad43e1a578c2e6a61012a8 2023-07-24 12:15:16,550 INFO [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(310): Store=0030755a278176ce5ddf3114bdcdb65b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:16,550 INFO [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:16,551 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2 2023-07-24 12:15:16,551 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2 2023-07-24 12:15:16,552 INFO [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 0030755a278176ce5ddf3114bdcdb65b columnFamilyName cf2 2023-07-24 12:15:16,558 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/026e0e33452a47efa5838eee95abb353 2023-07-24 12:15:16,562 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/456c894be60f41bf843e260fc392b6dc 2023-07-24 12:15:16,566 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/6890bc547ff04404b822ee57574a3f74 2023-07-24 12:15:16,573 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/77153bf8b3564490b09ec4f07cf58109 2023-07-24 12:15:16,577 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/8bbf73f57eb84adf9afd4d0424bcd4a1 2023-07-24 12:15:16,581 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/b880b546353140448fa623f389d94ed7 2023-07-24 12:15:16,584 DEBUG [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/d376133b6ec34bb4a5afb4b43a84255f 2023-07-24 12:15:16,584 INFO [StoreOpener-0030755a278176ce5ddf3114bdcdb65b-1] regionserver.HStore(310): Store=0030755a278176ce5ddf3114bdcdb65b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:16,584 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:16,585 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:16,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(602): Cleaned up old failed split transaction detritus: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.splits 2023-07-24 12:15:16,588 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRollbackAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:15:16,589 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:16,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 0030755a278176ce5ddf3114bdcdb65b; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65896580, jitterRate=-0.018064439296722412}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:15:16,590 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 0030755a278176ce5ddf3114bdcdb65b: 2023-07-24 12:15:16,590 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b., pid=48, masterSystemTime=1690200916516 2023-07-24 12:15:16,591 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:16,592 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 12:15:16,592 DEBUG [RS:0;jenkins-hbase6:41789-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:15:16,592 DEBUG [RS:0;jenkins-hbase6:41789-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:15:16,593 DEBUG [RS:0;jenkins-hbase6:41789-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-24 12:15:16,593 DEBUG [RS:0;jenkins-hbase6:41789-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-24 12:15:16,593 DEBUG [RS:0;jenkins-hbase6:41789-shortCompactions-0] regionserver.HStore(1912): 0030755a278176ce5ddf3114bdcdb65b/cf1 is initiating minor compaction (all files) 2023-07-24 12:15:16,593 DEBUG [RS:0;jenkins-hbase6:41789-longCompactions-0] regionserver.HStore(1912): 0030755a278176ce5ddf3114bdcdb65b/cf2 is initiating minor compaction (all files) 2023-07-24 12:15:16,593 INFO [RS:0;jenkins-hbase6:41789-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 0030755a278176ce5ddf3114bdcdb65b/cf1 in testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:16,593 INFO [RS:0;jenkins-hbase6:41789-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 0030755a278176ce5ddf3114bdcdb65b/cf2 in testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:16,593 INFO [RS:0;jenkins-hbase6:41789-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/bbf93197398c4190be29287d5e08cd51, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/21b67614581447b6abbaff9bffde0ad7, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/01a7666e16ed42a394a701ff42dc45e6, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/ff3d6ff223ad43e1a578c2e6a61012a8, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/69bebdfeb3144f59a9b55ee23b35a9e1, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/75a74fb05121409396d7ae1c3c6310bf, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/9b6acb4cd1ca40a2916f677d43ee62e2] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp, totalSize=34.9 K 2023-07-24 12:15:16,593 INFO [RS:0;jenkins-hbase6:41789-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/d376133b6ec34bb4a5afb4b43a84255f, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/8bbf73f57eb84adf9afd4d0424bcd4a1, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/6890bc547ff04404b822ee57574a3f74, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/b880b546353140448fa623f389d94ed7, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/456c894be60f41bf843e260fc392b6dc, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/77153bf8b3564490b09ec4f07cf58109, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/026e0e33452a47efa5838eee95abb353] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp, totalSize=34.9 K 2023-07-24 12:15:16,594 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:16,594 DEBUG [RS:0;jenkins-hbase6:41789-shortCompactions-0] compactions.Compactor(207): Compacting bbf93197398c4190be29287d5e08cd51, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690200909172 2023-07-24 12:15:16,594 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:16,594 DEBUG [RS:0;jenkins-hbase6:41789-longCompactions-0] compactions.Compactor(207): Compacting d376133b6ec34bb4a5afb4b43a84255f, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690200909172 2023-07-24 12:15:16,595 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=47 updating hbase:meta row=0030755a278176ce5ddf3114bdcdb65b, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:16,595 DEBUG [RS:0;jenkins-hbase6:41789-shortCompactions-0] compactions.Compactor(207): Compacting 21b67614581447b6abbaff9bffde0ad7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690200909365 2023-07-24 12:15:16,595 DEBUG [RS:0;jenkins-hbase6:41789-longCompactions-0] compactions.Compactor(207): Compacting 8bbf73f57eb84adf9afd4d0424bcd4a1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690200909365 2023-07-24 12:15:16,595 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200916594"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200916594"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200916594"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200916594"}]},"ts":"1690200916594"} 2023-07-24 12:15:16,595 DEBUG [RS:0;jenkins-hbase6:41789-shortCompactions-0] compactions.Compactor(207): Compacting 01a7666e16ed42a394a701ff42dc45e6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690200909563 2023-07-24 12:15:16,595 DEBUG [RS:0;jenkins-hbase6:41789-longCompactions-0] compactions.Compactor(207): Compacting 6890bc547ff04404b822ee57574a3f74, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690200909563 2023-07-24 12:15:16,595 DEBUG [RS:0;jenkins-hbase6:41789-shortCompactions-0] compactions.Compactor(207): Compacting ff3d6ff223ad43e1a578c2e6a61012a8, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690200909757 2023-07-24 12:15:16,596 DEBUG [RS:0;jenkins-hbase6:41789-longCompactions-0] compactions.Compactor(207): Compacting b880b546353140448fa623f389d94ed7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690200909757 2023-07-24 12:15:16,596 DEBUG [RS:0;jenkins-hbase6:41789-shortCompactions-0] compactions.Compactor(207): Compacting 69bebdfeb3144f59a9b55ee23b35a9e1, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690200909946 2023-07-24 12:15:16,596 DEBUG [RS:0;jenkins-hbase6:41789-longCompactions-0] compactions.Compactor(207): Compacting 456c894be60f41bf843e260fc392b6dc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690200909946 2023-07-24 12:15:16,596 DEBUG [RS:0;jenkins-hbase6:41789-shortCompactions-0] compactions.Compactor(207): Compacting 75a74fb05121409396d7ae1c3c6310bf, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690200910146 2023-07-24 12:15:16,596 DEBUG [RS:0;jenkins-hbase6:41789-longCompactions-0] compactions.Compactor(207): Compacting 77153bf8b3564490b09ec4f07cf58109, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690200910146 2023-07-24 12:15:16,597 DEBUG [RS:0;jenkins-hbase6:41789-shortCompactions-0] compactions.Compactor(207): Compacting 9b6acb4cd1ca40a2916f677d43ee62e2, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690200910338 2023-07-24 12:15:16,597 DEBUG [RS:0;jenkins-hbase6:41789-longCompactions-0] compactions.Compactor(207): Compacting 026e0e33452a47efa5838eee95abb353, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690200910338 2023-07-24 12:15:16,604 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=48, resume processing ppid=47 2023-07-24 12:15:16,604 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=48, ppid=47, state=SUCCESS; OpenRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 in 238 msec 2023-07-24 12:15:16,606 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN in 1.5620 sec 2023-07-24 12:15:16,628 INFO [RS:0;jenkins-hbase6:41789-shortCompactions-0] throttle.PressureAwareThroughputController(145): 0030755a278176ce5ddf3114bdcdb65b#cf1#compaction#51 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:16,628 INFO [RS:0;jenkins-hbase6:41789-longCompactions-0] throttle.PressureAwareThroughputController(145): 0030755a278176ce5ddf3114bdcdb65b#cf2#compaction#50 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:16,648 DEBUG [RS:0;jenkins-hbase6:41789-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf2/e8215f4983444c5aa9de27ec0fcaef91 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/e8215f4983444c5aa9de27ec0fcaef91 2023-07-24 12:15:16,653 INFO [RS:0;jenkins-hbase6:41789-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0030755a278176ce5ddf3114bdcdb65b/cf2 of 0030755a278176ce5ddf3114bdcdb65b into e8215f4983444c5aa9de27ec0fcaef91(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:16,653 DEBUG [RS:0;jenkins-hbase6:41789-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0030755a278176ce5ddf3114bdcdb65b: 2023-07-24 12:15:16,653 INFO [RS:0;jenkins-hbase6:41789-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b., storeName=0030755a278176ce5ddf3114bdcdb65b/cf2, priority=9, startTime=1690200916591; duration=0sec 2023-07-24 12:15:16,660 DEBUG [RS:0;jenkins-hbase6:41789-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:16,668 DEBUG [RS:0;jenkins-hbase6:41789-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/.tmp/cf1/98fed4f7a40f49d3a6b62426880929a2 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/98fed4f7a40f49d3a6b62426880929a2 2023-07-24 12:15:16,673 INFO [RS:0;jenkins-hbase6:41789-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 0030755a278176ce5ddf3114bdcdb65b/cf1 of 0030755a278176ce5ddf3114bdcdb65b into 98fed4f7a40f49d3a6b62426880929a2(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:16,673 DEBUG [RS:0;jenkins-hbase6:41789-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 0030755a278176ce5ddf3114bdcdb65b: 2023-07-24 12:15:16,673 INFO [RS:0;jenkins-hbase6:41789-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b., storeName=0030755a278176ce5ddf3114bdcdb65b/cf1, priority=9, startTime=1690200916591; duration=0sec 2023-07-24 12:15:16,673 DEBUG [RS:0;jenkins-hbase6:41789-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:16,713 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(335): procId=44 exception: org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested 2023-07-24 12:15:16,716 DEBUG [Listener at localhost/37279] hbase.HBaseTestingUtility(3430): Waiting until all regions of table testRollbackAndDoubleExecution get assigned. Timeout = 1500ms 2023-07-24 12:15:16,718 INFO [Listener at localhost/37279] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-07-24 12:15:16,720 INFO [Listener at localhost/37279] hbase.HBaseTestingUtility(3484): All regions for table testRollbackAndDoubleExecution assigned to meta. Checking AM states. 2023-07-24 12:15:16,721 INFO [Listener at localhost/37279] hbase.Waiter(180): Waiting up to [1,500] milli-secs(wait.for.ratio=[1]) 2023-07-24 12:15:16,721 INFO [Listener at localhost/37279] hbase.HBaseTestingUtility(3504): All regions for table testRollbackAndDoubleExecution assigned. 2023-07-24 12:15:16,740 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:15:16,741 INFO [Listener at localhost/37279] client.HBaseAdmin$15(890): Started disable of testRollbackAndDoubleExecution 2023-07-24 12:15:16,742 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$11(2418): Client=jenkins//172.31.2.20 disable testRollbackAndDoubleExecution 2023-07-24 12:15:16,743 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=49, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:16,745 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-24 12:15:16,745 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200916745"}]},"ts":"1690200916745"} 2023-07-24 12:15:16,747 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLING in hbase:meta 2023-07-24 12:15:16,748 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackAndDoubleExecution to state=DISABLING 2023-07-24 12:15:16,749 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN}] 2023-07-24 12:15:16,750 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=50, ppid=49, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN 2023-07-24 12:15:16,750 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:16,750 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200916750"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200916750"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200916750"}]},"ts":"1690200916750"} 2023-07-24 12:15:16,752 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=51, ppid=50, state=RUNNABLE; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488}] 2023-07-24 12:15:16,846 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-24 12:15:16,903 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:16,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 0030755a278176ce5ddf3114bdcdb65b, disabling compactions & flushes 2023-07-24 12:15:16,904 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:16,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:16,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. after waiting 0 ms 2023-07-24 12:15:16,904 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:16,904 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/bbf93197398c4190be29287d5e08cd51, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/21b67614581447b6abbaff9bffde0ad7, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/01a7666e16ed42a394a701ff42dc45e6, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/ff3d6ff223ad43e1a578c2e6a61012a8, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/69bebdfeb3144f59a9b55ee23b35a9e1, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/75a74fb05121409396d7ae1c3c6310bf, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/9b6acb4cd1ca40a2916f677d43ee62e2] to archive 2023-07-24 12:15:16,905 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:15:16,907 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/bbf93197398c4190be29287d5e08cd51 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/bbf93197398c4190be29287d5e08cd51 2023-07-24 12:15:16,908 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/21b67614581447b6abbaff9bffde0ad7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/21b67614581447b6abbaff9bffde0ad7 2023-07-24 12:15:16,909 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/01a7666e16ed42a394a701ff42dc45e6 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/01a7666e16ed42a394a701ff42dc45e6 2023-07-24 12:15:16,910 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/ff3d6ff223ad43e1a578c2e6a61012a8 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/ff3d6ff223ad43e1a578c2e6a61012a8 2023-07-24 12:15:16,912 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/69bebdfeb3144f59a9b55ee23b35a9e1 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/69bebdfeb3144f59a9b55ee23b35a9e1 2023-07-24 12:15:16,913 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/75a74fb05121409396d7ae1c3c6310bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/75a74fb05121409396d7ae1c3c6310bf 2023-07-24 12:15:16,914 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/9b6acb4cd1ca40a2916f677d43ee62e2 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/9b6acb4cd1ca40a2916f677d43ee62e2 2023-07-24 12:15:16,920 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/d376133b6ec34bb4a5afb4b43a84255f, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/8bbf73f57eb84adf9afd4d0424bcd4a1, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/6890bc547ff04404b822ee57574a3f74, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/b880b546353140448fa623f389d94ed7, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/456c894be60f41bf843e260fc392b6dc, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/77153bf8b3564490b09ec4f07cf58109, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/026e0e33452a47efa5838eee95abb353] to archive 2023-07-24 12:15:16,920 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:15:16,922 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/d376133b6ec34bb4a5afb4b43a84255f to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/d376133b6ec34bb4a5afb4b43a84255f 2023-07-24 12:15:16,923 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/8bbf73f57eb84adf9afd4d0424bcd4a1 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/8bbf73f57eb84adf9afd4d0424bcd4a1 2023-07-24 12:15:16,924 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/6890bc547ff04404b822ee57574a3f74 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/6890bc547ff04404b822ee57574a3f74 2023-07-24 12:15:16,926 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/b880b546353140448fa623f389d94ed7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/b880b546353140448fa623f389d94ed7 2023-07-24 12:15:16,927 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/456c894be60f41bf843e260fc392b6dc to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/456c894be60f41bf843e260fc392b6dc 2023-07-24 12:15:16,928 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/77153bf8b3564490b09ec4f07cf58109 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/77153bf8b3564490b09ec4f07cf58109 2023-07-24 12:15:16,930 DEBUG [StoreCloser-testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/026e0e33452a47efa5838eee95abb353 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/026e0e33452a47efa5838eee95abb353 2023-07-24 12:15:16,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 12:15:16,934 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:16,934 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b. 2023-07-24 12:15:16,935 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 0030755a278176ce5ddf3114bdcdb65b: 2023-07-24 12:15:16,936 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:16,937 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=50 updating hbase:meta row=0030755a278176ce5ddf3114bdcdb65b, regionState=CLOSED 2023-07-24 12:15:16,937 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200916937"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200916937"}]},"ts":"1690200916937"} 2023-07-24 12:15:16,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=51, resume processing ppid=50 2023-07-24 12:15:16,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=51, ppid=50, state=SUCCESS; CloseRegionProcedure 0030755a278176ce5ddf3114bdcdb65b, server=jenkins-hbase6.apache.org,41789,1690200894488 in 186 msec 2023-07-24 12:15:16,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=50, resume processing ppid=49 2023-07-24 12:15:16,942 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=50, ppid=49, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, UNASSIGN in 191 msec 2023-07-24 12:15:16,942 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200916942"}]},"ts":"1690200916942"} 2023-07-24 12:15:16,944 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackAndDoubleExecution, state=DISABLED in hbase:meta 2023-07-24 12:15:16,945 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackAndDoubleExecution to state=DISABLED 2023-07-24 12:15:16,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution in 203 msec 2023-07-24 12:15:17,047 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=49 2023-07-24 12:15:17,048 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackAndDoubleExecution, procId: 49 completed 2023-07-24 12:15:17,048 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$5(2228): Client=jenkins//172.31.2.20 delete testRollbackAndDoubleExecution 2023-07-24 12:15:17,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:17,051 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=52, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:17,052 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=52, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:17,052 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-07-24 12:15:17,055 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:17,057 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/recovered.edits] 2023-07-24 12:15:17,059 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/98fed4f7a40f49d3a6b62426880929a2 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf1/98fed4f7a40f49d3a6b62426880929a2 2023-07-24 12:15:17,061 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/e8215f4983444c5aa9de27ec0fcaef91 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/cf2/e8215f4983444c5aa9de27ec0fcaef91 2023-07-24 12:15:17,064 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/recovered.edits/90.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b/recovered.edits/90.seqid 2023-07-24 12:15:17,065 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackAndDoubleExecution/0030755a278176ce5ddf3114bdcdb65b 2023-07-24 12:15:17,065 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackAndDoubleExecution regions 2023-07-24 12:15:17,068 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=52, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:17,069 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackAndDoubleExecution from hbase:meta 2023-07-24 12:15:17,071 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackAndDoubleExecution' descriptor. 2023-07-24 12:15:17,072 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=52, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:17,072 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackAndDoubleExecution' from region states. 2023-07-24 12:15:17,072 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200917072"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:17,074 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-24 12:15:17,074 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 0030755a278176ce5ddf3114bdcdb65b, NAME => 'testRollbackAndDoubleExecution,,1690200908667.0030755a278176ce5ddf3114bdcdb65b.', STARTKEY => '', ENDKEY => ''}] 2023-07-24 12:15:17,074 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackAndDoubleExecution' as deleted. 2023-07-24 12:15:17,074 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690200917074"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:17,075 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackAndDoubleExecution state from META 2023-07-24 12:15:17,077 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=52, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:15:17,078 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution in 29 msec 2023-07-24 12:15:17,153 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=52 2023-07-24 12:15:17,153 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackAndDoubleExecution, procId: 52 completed 2023-07-24 12:15:17,170 INFO [Listener at localhost/37279] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackAndDoubleExecution Thread=401 (was 400) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/dfs/data/data4/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:44594 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:60550 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:45670 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:60596 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:44276 [Waiting for operation #46] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:51456 [Waiting for operation #25] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:60534 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:44618 [Waiting for operation #29] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-9 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:45664 [Waiting for operation #9] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:51280 [Waiting for operation #38] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:60582 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:0;jenkins-hbase6:41789-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:37536 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:38544 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:60612 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:60566 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: Async disk worker #0 for volume /home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/dfs/data/data3/current sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-584534869_17 at /127.0.0.1:39758 [Waiting for operation #39] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=871 (was 872), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=341 (was 310) - SystemLoadAverage LEAK? -, ProcessCount=169 (was 169), AvailableMemoryMB=1574 (was 1693) 2023-07-24 12:15:17,184 INFO [Listener at localhost/37279] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=401, OpenFileDescriptor=871, MaxFileDescriptor=60000, SystemLoadAverage=341, ProcessCount=169, AvailableMemoryMB=1573 2023-07-24 12:15:17,184 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:15:17,188 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 12:15:17,188 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(492): Client=jenkins//172.31.2.20 set balanceSwitch=false 2023-07-24 12:15:17,189 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:15:17,191 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 12:15:17,192 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 12:15:17,193 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5 2023-07-24 12:15:17,194 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5 empty. 2023-07-24 12:15:17,194 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5 2023-07-24 12:15:17,194 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-07-24 12:15:17,204 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/.tabledesc/.tableinfo.0000000001 2023-07-24 12:15:17,205 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(7675): creating {ENCODED => 6ee4b5a3446811147857e7c4855e0fc5, NAME => 'testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionNoStoreFile', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp 2023-07-24 12:15:17,212 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:17,212 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1604): Closing 6ee4b5a3446811147857e7c4855e0fc5, disabling compactions & flushes 2023-07-24 12:15:17,212 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5. 2023-07-24 12:15:17,212 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5. 2023-07-24 12:15:17,212 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5. after waiting 0 ms 2023-07-24 12:15:17,212 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5. 2023-07-24 12:15:17,212 INFO [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5. 2023-07-24 12:15:17,212 DEBUG [RegionOpenAndInit-testSplitTableRegionNoStoreFile-pool-0] regionserver.HRegion(1558): Region close journal for 6ee4b5a3446811147857e7c4855e0fc5: 2023-07-24 12:15:17,215 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 12:15:17,216 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1690200917216"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200917216"}]},"ts":"1690200917216"} 2023-07-24 12:15:17,218 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 12:15:17,219 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 12:15:17,220 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200917219"}]},"ts":"1690200917219"} 2023-07-24 12:15:17,221 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLING in hbase:meta 2023-07-24 12:15:17,224 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase6.apache.org=0} racks are {/default-rack=0} 2023-07-24 12:15:17,225 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 12:15:17,225 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 12:15:17,225 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 12:15:17,225 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 12:15:17,226 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ee4b5a3446811147857e7c4855e0fc5, ASSIGN}] 2023-07-24 12:15:17,227 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ee4b5a3446811147857e7c4855e0fc5, ASSIGN 2023-07-24 12:15:17,228 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=54, ppid=53, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ee4b5a3446811147857e7c4855e0fc5, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,36983,1690200894572; forceNewPlan=false, retain=false 2023-07-24 12:15:17,378 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 12:15:17,379 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=6ee4b5a3446811147857e7c4855e0fc5, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:17,379 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1690200917379"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200917379"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200917379"}]},"ts":"1690200917379"} 2023-07-24 12:15:17,381 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=55, ppid=54, state=RUNNABLE; OpenRegionProcedure 6ee4b5a3446811147857e7c4855e0fc5, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:17,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5. 2023-07-24 12:15:17,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6ee4b5a3446811147857e7c4855e0fc5, NAME => 'testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5.', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:15:17,536 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:17,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 6ee4b5a3446811147857e7c4855e0fc5 2023-07-24 12:15:17,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:17,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 6ee4b5a3446811147857e7c4855e0fc5 2023-07-24 12:15:17,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 6ee4b5a3446811147857e7c4855e0fc5 2023-07-24 12:15:17,538 INFO [StoreOpener-6ee4b5a3446811147857e7c4855e0fc5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6ee4b5a3446811147857e7c4855e0fc5 2023-07-24 12:15:17,539 DEBUG [StoreOpener-6ee4b5a3446811147857e7c4855e0fc5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5/cf1 2023-07-24 12:15:17,539 DEBUG [StoreOpener-6ee4b5a3446811147857e7c4855e0fc5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5/cf1 2023-07-24 12:15:17,540 INFO [StoreOpener-6ee4b5a3446811147857e7c4855e0fc5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ee4b5a3446811147857e7c4855e0fc5 columnFamilyName cf1 2023-07-24 12:15:17,540 INFO [StoreOpener-6ee4b5a3446811147857e7c4855e0fc5-1] regionserver.HStore(310): Store=6ee4b5a3446811147857e7c4855e0fc5/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:17,540 INFO [StoreOpener-6ee4b5a3446811147857e7c4855e0fc5-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6ee4b5a3446811147857e7c4855e0fc5 2023-07-24 12:15:17,542 DEBUG [StoreOpener-6ee4b5a3446811147857e7c4855e0fc5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5/cf2 2023-07-24 12:15:17,542 DEBUG [StoreOpener-6ee4b5a3446811147857e7c4855e0fc5-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5/cf2 2023-07-24 12:15:17,542 INFO [StoreOpener-6ee4b5a3446811147857e7c4855e0fc5-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6ee4b5a3446811147857e7c4855e0fc5 columnFamilyName cf2 2023-07-24 12:15:17,543 INFO [StoreOpener-6ee4b5a3446811147857e7c4855e0fc5-1] regionserver.HStore(310): Store=6ee4b5a3446811147857e7c4855e0fc5/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:17,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5 2023-07-24 12:15:17,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5 2023-07-24 12:15:17,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:15:17,547 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 6ee4b5a3446811147857e7c4855e0fc5 2023-07-24 12:15:17,549 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 12:15:17,550 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 6ee4b5a3446811147857e7c4855e0fc5; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67804215, jitterRate=0.010361537337303162}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:15:17,550 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 6ee4b5a3446811147857e7c4855e0fc5: 2023-07-24 12:15:17,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5., pid=55, masterSystemTime=1690200917533 2023-07-24 12:15:17,552 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5. 2023-07-24 12:15:17,552 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5. 2023-07-24 12:15:17,553 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=54 updating hbase:meta row=6ee4b5a3446811147857e7c4855e0fc5, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:17,553 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1690200917553"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200917553"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200917553"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200917553"}]},"ts":"1690200917553"} 2023-07-24 12:15:17,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=55, resume processing ppid=54 2023-07-24 12:15:17,556 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=55, ppid=54, state=SUCCESS; OpenRegionProcedure 6ee4b5a3446811147857e7c4855e0fc5, server=jenkins-hbase6.apache.org,36983,1690200894572 in 173 msec 2023-07-24 12:15:17,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=54, resume processing ppid=53 2023-07-24 12:15:17,558 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=54, ppid=53, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ee4b5a3446811147857e7c4855e0fc5, ASSIGN in 331 msec 2023-07-24 12:15:17,559 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 12:15:17,559 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200917559"}]},"ts":"1690200917559"} 2023-07-24 12:15:17,560 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=ENABLED in hbase:meta 2023-07-24 12:15:17,562 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=53, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionNoStoreFile execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 12:15:17,563 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile in 374 msec 2023-07-24 12:15:17,690 DEBUG [Listener at localhost/37279] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:17,692 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:17,693 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:17,693 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:17,693 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=56, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:17,695 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ee4b5a3446811147857e7c4855e0fc5, UNASSIGN}] 2023-07-24 12:15:17,696 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=57, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ee4b5a3446811147857e7c4855e0fc5, UNASSIGN 2023-07-24 12:15:17,696 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=6ee4b5a3446811147857e7c4855e0fc5, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:17,697 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1690200917696"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200917696"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200917696"}]},"ts":"1690200917696"} 2023-07-24 12:15:17,698 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=58, ppid=57, state=RUNNABLE; CloseRegionProcedure 6ee4b5a3446811147857e7c4855e0fc5, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:17,850 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 6ee4b5a3446811147857e7c4855e0fc5 2023-07-24 12:15:17,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 6ee4b5a3446811147857e7c4855e0fc5, disabling compactions & flushes 2023-07-24 12:15:17,851 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5. 2023-07-24 12:15:17,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5. 2023-07-24 12:15:17,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5. after waiting 0 ms 2023-07-24 12:15:17,851 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5. 2023-07-24 12:15:17,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 12:15:17,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:17,855 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5. 2023-07-24 12:15:17,855 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 6ee4b5a3446811147857e7c4855e0fc5: 2023-07-24 12:15:17,857 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 6ee4b5a3446811147857e7c4855e0fc5 2023-07-24 12:15:17,857 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=57 updating hbase:meta row=6ee4b5a3446811147857e7c4855e0fc5, regionState=CLOSED 2023-07-24 12:15:17,857 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1690200917857"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200917857"}]},"ts":"1690200917857"} 2023-07-24 12:15:17,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=58, resume processing ppid=57 2023-07-24 12:15:17,860 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=58, ppid=57, state=SUCCESS; CloseRegionProcedure 6ee4b5a3446811147857e7c4855e0fc5, server=jenkins-hbase6.apache.org,36983,1690200894572 in 161 msec 2023-07-24 12:15:17,861 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=57, resume processing ppid=56 2023-07-24 12:15:17,862 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=57, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=6ee4b5a3446811147857e7c4855e0fc5, UNASSIGN in 165 msec 2023-07-24 12:15:17,887 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-07-24 12:15:17,889 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=-1 2023-07-24 12:15:17,890 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5.","families":{"info":[{"qualifier":"regioninfo","vlen":65,"tag":[],"timestamp":"1690200917890"},{"qualifier":"splitA","vlen":67,"tag":[],"timestamp":"1690200917890"},{"qualifier":"splitB","vlen":67,"tag":[],"timestamp":"1690200917890"}]},"ts":"1690200917890"} 2023-07-24 12:15:17,890 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690200917890"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200917890"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200917890"}]},"ts":"1690200917890"} 2023-07-24 12:15:17,890 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690200917890"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200917890"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200917890"}]},"ts":"1690200917890"} 2023-07-24 12:15:17,893 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=f5462d2626c742685e8ed96fb350e173, ASSIGN}, {pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=868f7f720d1f1b0e94a32099a89c6854, ASSIGN}] 2023-07-24 12:15:17,894 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=868f7f720d1f1b0e94a32099a89c6854, ASSIGN 2023-07-24 12:15:17,895 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=60, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=868f7f720d1f1b0e94a32099a89c6854, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase6.apache.org,36983,1690200894572; forceNewPlan=false, retain=false 2023-07-24 12:15:17,895 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=f5462d2626c742685e8ed96fb350e173, ASSIGN 2023-07-24 12:15:17,896 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=59, ppid=56, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=f5462d2626c742685e8ed96fb350e173, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase6.apache.org,36983,1690200894572; forceNewPlan=false, retain=false 2023-07-24 12:15:18,045 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-24 12:15:18,047 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=f5462d2626c742685e8ed96fb350e173, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:18,047 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690200918047"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200918047"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200918047"}]},"ts":"1690200918047"} 2023-07-24 12:15:18,049 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=61, ppid=59, state=RUNNABLE; OpenRegionProcedure f5462d2626c742685e8ed96fb350e173, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:18,052 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=868f7f720d1f1b0e94a32099a89c6854, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:18,052 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690200918052"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200918052"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200918052"}]},"ts":"1690200918052"} 2023-07-24 12:15:18,053 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=62, ppid=60, state=RUNNABLE; OpenRegionProcedure 868f7f720d1f1b0e94a32099a89c6854, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:18,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173. 2023-07-24 12:15:18,204 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => f5462d2626c742685e8ed96fb350e173, NAME => 'testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173.', STARTKEY => '', ENDKEY => '41'} 2023-07-24 12:15:18,205 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:18,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile f5462d2626c742685e8ed96fb350e173 2023-07-24 12:15:18,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:18,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for f5462d2626c742685e8ed96fb350e173 2023-07-24 12:15:18,205 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for f5462d2626c742685e8ed96fb350e173 2023-07-24 12:15:18,206 INFO [StoreOpener-f5462d2626c742685e8ed96fb350e173-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region f5462d2626c742685e8ed96fb350e173 2023-07-24 12:15:18,207 DEBUG [StoreOpener-f5462d2626c742685e8ed96fb350e173-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173/cf1 2023-07-24 12:15:18,207 DEBUG [StoreOpener-f5462d2626c742685e8ed96fb350e173-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173/cf1 2023-07-24 12:15:18,208 INFO [StoreOpener-f5462d2626c742685e8ed96fb350e173-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f5462d2626c742685e8ed96fb350e173 columnFamilyName cf1 2023-07-24 12:15:18,208 INFO [StoreOpener-f5462d2626c742685e8ed96fb350e173-1] regionserver.HStore(310): Store=f5462d2626c742685e8ed96fb350e173/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:18,208 INFO [StoreOpener-f5462d2626c742685e8ed96fb350e173-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region f5462d2626c742685e8ed96fb350e173 2023-07-24 12:15:18,210 DEBUG [StoreOpener-f5462d2626c742685e8ed96fb350e173-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173/cf2 2023-07-24 12:15:18,210 DEBUG [StoreOpener-f5462d2626c742685e8ed96fb350e173-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173/cf2 2023-07-24 12:15:18,210 INFO [StoreOpener-f5462d2626c742685e8ed96fb350e173-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region f5462d2626c742685e8ed96fb350e173 columnFamilyName cf2 2023-07-24 12:15:18,210 INFO [StoreOpener-f5462d2626c742685e8ed96fb350e173-1] regionserver.HStore(310): Store=f5462d2626c742685e8ed96fb350e173/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:18,211 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173 2023-07-24 12:15:18,212 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173 2023-07-24 12:15:18,214 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:15:18,215 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for f5462d2626c742685e8ed96fb350e173 2023-07-24 12:15:18,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened f5462d2626c742685e8ed96fb350e173; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70970373, jitterRate=0.05754096806049347}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:15:18,216 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for f5462d2626c742685e8ed96fb350e173: 2023-07-24 12:15:18,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173., pid=61, masterSystemTime=1690200918201 2023-07-24 12:15:18,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173. 2023-07-24 12:15:18,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173. 2023-07-24 12:15:18,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854. 2023-07-24 12:15:18,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 868f7f720d1f1b0e94a32099a89c6854, NAME => 'testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854.', STARTKEY => '41', ENDKEY => ''} 2023-07-24 12:15:18,218 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=59 updating hbase:meta row=f5462d2626c742685e8ed96fb350e173, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:18,218 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:18,218 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690200918218"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200918218"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200918218"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200918218"}]},"ts":"1690200918218"} 2023-07-24 12:15:18,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionNoStoreFile 868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:18,218 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:18,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:18,219 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:18,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=61, resume processing ppid=59 2023-07-24 12:15:18,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=61, ppid=59, state=SUCCESS; OpenRegionProcedure f5462d2626c742685e8ed96fb350e173, server=jenkins-hbase6.apache.org,36983,1690200894572 in 171 msec 2023-07-24 12:15:18,223 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=59, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=f5462d2626c742685e8ed96fb350e173, ASSIGN in 329 msec 2023-07-24 12:15:18,224 INFO [StoreOpener-868f7f720d1f1b0e94a32099a89c6854-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:18,225 DEBUG [StoreOpener-868f7f720d1f1b0e94a32099a89c6854-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854/cf1 2023-07-24 12:15:18,225 DEBUG [StoreOpener-868f7f720d1f1b0e94a32099a89c6854-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854/cf1 2023-07-24 12:15:18,225 INFO [StoreOpener-868f7f720d1f1b0e94a32099a89c6854-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 868f7f720d1f1b0e94a32099a89c6854 columnFamilyName cf1 2023-07-24 12:15:18,225 INFO [StoreOpener-868f7f720d1f1b0e94a32099a89c6854-1] regionserver.HStore(310): Store=868f7f720d1f1b0e94a32099a89c6854/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:18,226 INFO [StoreOpener-868f7f720d1f1b0e94a32099a89c6854-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:18,227 DEBUG [StoreOpener-868f7f720d1f1b0e94a32099a89c6854-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854/cf2 2023-07-24 12:15:18,227 DEBUG [StoreOpener-868f7f720d1f1b0e94a32099a89c6854-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854/cf2 2023-07-24 12:15:18,227 INFO [StoreOpener-868f7f720d1f1b0e94a32099a89c6854-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 868f7f720d1f1b0e94a32099a89c6854 columnFamilyName cf2 2023-07-24 12:15:18,227 INFO [StoreOpener-868f7f720d1f1b0e94a32099a89c6854-1] regionserver.HStore(310): Store=868f7f720d1f1b0e94a32099a89c6854/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:18,228 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:18,229 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:18,231 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionNoStoreFile descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:15:18,232 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:18,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 868f7f720d1f1b0e94a32099a89c6854; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=62392611, jitterRate=-0.07027764618396759}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:15:18,233 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 868f7f720d1f1b0e94a32099a89c6854: 2023-07-24 12:15:18,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854., pid=62, masterSystemTime=1690200918201 2023-07-24 12:15:18,235 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854. 2023-07-24 12:15:18,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854. 2023-07-24 12:15:18,235 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=60 updating hbase:meta row=868f7f720d1f1b0e94a32099a89c6854, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:18,235 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690200918235"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200918235"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200918235"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200918235"}]},"ts":"1690200918235"} 2023-07-24 12:15:18,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=62, resume processing ppid=60 2023-07-24 12:15:18,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=62, ppid=60, state=SUCCESS; OpenRegionProcedure 868f7f720d1f1b0e94a32099a89c6854, server=jenkins-hbase6.apache.org,36983,1690200894572 in 183 msec 2023-07-24 12:15:18,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=60, resume processing ppid=56 2023-07-24 12:15:18,240 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=60, ppid=56, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=868f7f720d1f1b0e94a32099a89c6854, ASSIGN in 345 msec 2023-07-24 12:15:18,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 in 549 msec 2023-07-24 12:15:18,446 DEBUG [Listener at localhost/37279] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-24 12:15:18,449 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:15:18,451 INFO [Listener at localhost/37279] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionNoStoreFile 2023-07-24 12:15:18,451 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$11(2418): Client=jenkins//172.31.2.20 disable testSplitTableRegionNoStoreFile 2023-07-24 12:15:18,452 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=63, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:15:18,455 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-07-24 12:15:18,455 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200918455"}]},"ts":"1690200918455"} 2023-07-24 12:15:18,456 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLING in hbase:meta 2023-07-24 12:15:18,458 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionNoStoreFile to state=DISABLING 2023-07-24 12:15:18,459 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=f5462d2626c742685e8ed96fb350e173, UNASSIGN}, {pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=868f7f720d1f1b0e94a32099a89c6854, UNASSIGN}] 2023-07-24 12:15:18,460 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=65, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=868f7f720d1f1b0e94a32099a89c6854, UNASSIGN 2023-07-24 12:15:18,460 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=868f7f720d1f1b0e94a32099a89c6854, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:18,460 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690200918460"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200918460"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200918460"}]},"ts":"1690200918460"} 2023-07-24 12:15:18,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=66, ppid=65, state=RUNNABLE; CloseRegionProcedure 868f7f720d1f1b0e94a32099a89c6854, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:18,463 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=64, ppid=63, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=f5462d2626c742685e8ed96fb350e173, UNASSIGN 2023-07-24 12:15:18,464 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=f5462d2626c742685e8ed96fb350e173, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:18,464 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690200918464"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200918464"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200918464"}]},"ts":"1690200918464"} 2023-07-24 12:15:18,465 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=67, ppid=64, state=RUNNABLE; CloseRegionProcedure f5462d2626c742685e8ed96fb350e173, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:18,556 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-07-24 12:15:18,613 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close f5462d2626c742685e8ed96fb350e173 2023-07-24 12:15:18,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing f5462d2626c742685e8ed96fb350e173, disabling compactions & flushes 2023-07-24 12:15:18,614 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173. 2023-07-24 12:15:18,614 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173. 2023-07-24 12:15:18,615 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173. after waiting 0 ms 2023-07-24 12:15:18,615 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173. 2023-07-24 12:15:18,619 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-07-24 12:15:18,619 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:18,620 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173. 2023-07-24 12:15:18,620 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for f5462d2626c742685e8ed96fb350e173: 2023-07-24 12:15:18,621 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed f5462d2626c742685e8ed96fb350e173 2023-07-24 12:15:18,621 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:18,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 868f7f720d1f1b0e94a32099a89c6854, disabling compactions & flushes 2023-07-24 12:15:18,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854. 2023-07-24 12:15:18,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854. 2023-07-24 12:15:18,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854. after waiting 0 ms 2023-07-24 12:15:18,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854. 2023-07-24 12:15:18,623 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=64 updating hbase:meta row=f5462d2626c742685e8ed96fb350e173, regionState=CLOSED 2023-07-24 12:15:18,623 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690200918622"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200918622"}]},"ts":"1690200918622"} 2023-07-24 12:15:18,626 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=67, resume processing ppid=64 2023-07-24 12:15:18,626 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=67, ppid=64, state=SUCCESS; CloseRegionProcedure f5462d2626c742685e8ed96fb350e173, server=jenkins-hbase6.apache.org,36983,1690200894572 in 159 msec 2023-07-24 12:15:18,627 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-07-24 12:15:18,628 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:18,628 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854. 2023-07-24 12:15:18,628 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 868f7f720d1f1b0e94a32099a89c6854: 2023-07-24 12:15:18,628 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=64, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=f5462d2626c742685e8ed96fb350e173, UNASSIGN in 168 msec 2023-07-24 12:15:18,629 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:18,629 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=65 updating hbase:meta row=868f7f720d1f1b0e94a32099a89c6854, regionState=CLOSED 2023-07-24 12:15:18,630 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690200918629"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200918629"}]},"ts":"1690200918629"} 2023-07-24 12:15:18,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=66, resume processing ppid=65 2023-07-24 12:15:18,632 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=66, ppid=65, state=SUCCESS; CloseRegionProcedure 868f7f720d1f1b0e94a32099a89c6854, server=jenkins-hbase6.apache.org,36983,1690200894572 in 170 msec 2023-07-24 12:15:18,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=65, resume processing ppid=63 2023-07-24 12:15:18,634 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=65, ppid=63, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionNoStoreFile, region=868f7f720d1f1b0e94a32099a89c6854, UNASSIGN in 173 msec 2023-07-24 12:15:18,635 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200918634"}]},"ts":"1690200918634"} 2023-07-24 12:15:18,636 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionNoStoreFile, state=DISABLED in hbase:meta 2023-07-24 12:15:18,637 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionNoStoreFile to state=DISABLED 2023-07-24 12:15:18,639 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile in 187 msec 2023-07-24 12:15:18,757 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=63 2023-07-24 12:15:18,757 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 63 completed 2023-07-24 12:15:18,758 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$5(2228): Client=jenkins//172.31.2.20 delete testSplitTableRegionNoStoreFile 2023-07-24 12:15:18,759 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:15:18,760 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=68, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:15:18,761 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=68, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:15:18,761 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-24 12:15:18,767 DEBUG [HFileArchiver-3] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5 2023-07-24 12:15:18,767 DEBUG [HFileArchiver-6] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173 2023-07-24 12:15:18,767 DEBUG [HFileArchiver-7] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:18,770 DEBUG [HFileArchiver-6] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173/recovered.edits] 2023-07-24 12:15:18,770 DEBUG [HFileArchiver-3] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5/recovered.edits] 2023-07-24 12:15:18,771 DEBUG [HFileArchiver-7] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854/recovered.edits] 2023-07-24 12:15:18,778 DEBUG [HFileArchiver-6] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173/recovered.edits/7.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173/recovered.edits/7.seqid 2023-07-24 12:15:18,778 DEBUG [HFileArchiver-3] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5/recovered.edits/4.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5/recovered.edits/4.seqid 2023-07-24 12:15:18,778 DEBUG [HFileArchiver-6] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/f5462d2626c742685e8ed96fb350e173 2023-07-24 12:15:18,779 DEBUG [HFileArchiver-7] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854/recovered.edits/7.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854/recovered.edits/7.seqid 2023-07-24 12:15:18,779 DEBUG [HFileArchiver-3] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/6ee4b5a3446811147857e7c4855e0fc5 2023-07-24 12:15:18,779 DEBUG [HFileArchiver-7] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionNoStoreFile/868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:15:18,779 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionNoStoreFile regions 2023-07-24 12:15:18,782 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=68, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:15:18,784 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionNoStoreFile from hbase:meta 2023-07-24 12:15:18,785 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionNoStoreFile' descriptor. 2023-07-24 12:15:18,786 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=68, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:15:18,786 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionNoStoreFile' from region states. 2023-07-24 12:15:18,787 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200918786"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:18,787 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200918786"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:18,787 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200918786"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:18,788 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-24 12:15:18,788 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 6ee4b5a3446811147857e7c4855e0fc5, NAME => 'testSplitTableRegionNoStoreFile,,1690200917188.6ee4b5a3446811147857e7c4855e0fc5.', STARTKEY => '', ENDKEY => ''}, {ENCODED => f5462d2626c742685e8ed96fb350e173, NAME => 'testSplitTableRegionNoStoreFile,,1690200917691.f5462d2626c742685e8ed96fb350e173.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 868f7f720d1f1b0e94a32099a89c6854, NAME => 'testSplitTableRegionNoStoreFile,41,1690200917691.868f7f720d1f1b0e94a32099a89c6854.', STARTKEY => '41', ENDKEY => ''}] 2023-07-24 12:15:18,788 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionNoStoreFile' as deleted. 2023-07-24 12:15:18,788 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionNoStoreFile","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690200918788"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:18,790 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionNoStoreFile state from META 2023-07-24 12:15:18,791 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=68, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:15:18,792 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile in 33 msec 2023-07-24 12:15:18,862 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=68 2023-07-24 12:15:18,862 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionNoStoreFile, procId: 68 completed 2023-07-24 12:15:18,877 INFO [Listener at localhost/37279] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionNoStoreFile Thread=405 (was 401) Potentially hanging thread: hconnection-0x3d545144-shared-pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1812904278_17 at /127.0.0.1:51456 [Waiting for operation #26] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-1812904278_17 at /127.0.0.1:45664 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-metaLookup-shared--pool-7 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-metaLookup-shared--pool-8 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=871 (was 871), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=330 (was 341), ProcessCount=169 (was 169), AvailableMemoryMB=1572 (was 1573) 2023-07-24 12:15:18,890 INFO [Listener at localhost/37279] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=405, OpenFileDescriptor=871, MaxFileDescriptor=60000, SystemLoadAverage=330, ProcessCount=169, AvailableMemoryMB=1572 2023-07-24 12:15:18,891 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:15:18,893 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 12:15:18,893 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(492): Client=jenkins//172.31.2.20 set balanceSwitch=false 2023-07-24 12:15:18,894 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:18,896 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 12:15:18,897 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 12:15:18,899 DEBUG [HFileArchiver-1] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:18,899 DEBUG [HFileArchiver-1] backup.HFileArchiver(153): Directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7 empty. 2023-07-24 12:15:18,900 DEBUG [HFileArchiver-1] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:18,900 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-07-24 12:15:18,915 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/.tabledesc/.tableinfo.0000000001 2023-07-24 12:15:18,916 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => ed62c139047156182f9c4cdbe34edfa7, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionDeletedRowsDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp 2023-07-24 12:15:18,925 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:18,925 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1604): Closing ed62c139047156182f9c4cdbe34edfa7, disabling compactions & flushes 2023-07-24 12:15:18,925 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:18,925 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:18,925 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. after waiting 0 ms 2023-07-24 12:15:18,925 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:18,925 INFO [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:18,925 DEBUG [RegionOpenAndInit-testSplitTableRegionDeletedRowsDaughter-pool-0] regionserver.HRegion(1558): Region close journal for ed62c139047156182f9c4cdbe34edfa7: 2023-07-24 12:15:18,928 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 12:15:18,928 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1690200918928"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200918928"}]},"ts":"1690200918928"} 2023-07-24 12:15:18,930 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 12:15:18,931 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 12:15:18,931 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200918931"}]},"ts":"1690200918931"} 2023-07-24 12:15:18,932 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLING in hbase:meta 2023-07-24 12:15:18,936 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase6.apache.org=0} racks are {/default-rack=0} 2023-07-24 12:15:18,937 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 12:15:18,937 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 12:15:18,937 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 12:15:18,937 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 12:15:18,937 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ed62c139047156182f9c4cdbe34edfa7, ASSIGN}] 2023-07-24 12:15:18,938 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ed62c139047156182f9c4cdbe34edfa7, ASSIGN 2023-07-24 12:15:18,939 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=70, ppid=69, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ed62c139047156182f9c4cdbe34edfa7, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,34575,1690200894532; forceNewPlan=false, retain=false 2023-07-24 12:15:19,089 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 12:15:19,090 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=ed62c139047156182f9c4cdbe34edfa7, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,090 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1690200919090"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200919090"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200919090"}]},"ts":"1690200919090"} 2023-07-24 12:15:19,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=71, ppid=70, state=RUNNABLE; OpenRegionProcedure ed62c139047156182f9c4cdbe34edfa7, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:15:19,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:19,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ed62c139047156182f9c4cdbe34edfa7, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7.', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:15:19,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:19,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:19,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:19,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:19,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:19,253 INFO [StoreOpener-ed62c139047156182f9c4cdbe34edfa7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:19,256 DEBUG [StoreOpener-ed62c139047156182f9c4cdbe34edfa7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1 2023-07-24 12:15:19,256 DEBUG [StoreOpener-ed62c139047156182f9c4cdbe34edfa7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1 2023-07-24 12:15:19,256 INFO [StoreOpener-ed62c139047156182f9c4cdbe34edfa7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ed62c139047156182f9c4cdbe34edfa7 columnFamilyName cf1 2023-07-24 12:15:19,257 INFO [StoreOpener-ed62c139047156182f9c4cdbe34edfa7-1] regionserver.HStore(310): Store=ed62c139047156182f9c4cdbe34edfa7/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:19,257 INFO [StoreOpener-ed62c139047156182f9c4cdbe34edfa7-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:19,258 DEBUG [StoreOpener-ed62c139047156182f9c4cdbe34edfa7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2 2023-07-24 12:15:19,259 DEBUG [StoreOpener-ed62c139047156182f9c4cdbe34edfa7-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2 2023-07-24 12:15:19,259 INFO [StoreOpener-ed62c139047156182f9c4cdbe34edfa7-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ed62c139047156182f9c4cdbe34edfa7 columnFamilyName cf2 2023-07-24 12:15:19,260 INFO [StoreOpener-ed62c139047156182f9c4cdbe34edfa7-1] regionserver.HStore(310): Store=ed62c139047156182f9c4cdbe34edfa7/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:19,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:19,261 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:19,263 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:15:19,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:19,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 12:15:19,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened ed62c139047156182f9c4cdbe34edfa7; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63802204, jitterRate=-0.04927307367324829}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:15:19,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for ed62c139047156182f9c4cdbe34edfa7: 2023-07-24 12:15:19,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7., pid=71, masterSystemTime=1690200919245 2023-07-24 12:15:19,270 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:19,270 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:19,270 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=70 updating hbase:meta row=ed62c139047156182f9c4cdbe34edfa7, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,270 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1690200919270"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200919270"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200919270"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200919270"}]},"ts":"1690200919270"} 2023-07-24 12:15:19,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=71, resume processing ppid=70 2023-07-24 12:15:19,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=71, ppid=70, state=SUCCESS; OpenRegionProcedure ed62c139047156182f9c4cdbe34edfa7, server=jenkins-hbase6.apache.org,34575,1690200894532 in 180 msec 2023-07-24 12:15:19,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=70, resume processing ppid=69 2023-07-24 12:15:19,275 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=70, ppid=69, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ed62c139047156182f9c4cdbe34edfa7, ASSIGN in 337 msec 2023-07-24 12:15:19,276 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 12:15:19,276 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200919276"}]},"ts":"1690200919276"} 2023-07-24 12:15:19,277 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=ENABLED in hbase:meta 2023-07-24 12:15:19,279 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=69, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 12:15:19,280 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 385 msec 2023-07-24 12:15:19,397 DEBUG [Listener at localhost/37279] ipc.RpcConnection(124): Using SIMPLE authentication for service=ClientService, sasl=false 2023-07-24 12:15:19,400 INFO [RS-EventLoopGroup-4-1] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:46684, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=ClientService 2023-07-24 12:15:19,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:19,408 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,408 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:19,408 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,408 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 12:15:19,408 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 12:15:19,408 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,408 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,409 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,409 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,409 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,409 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,410 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,410 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:19,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,410 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,410 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,410 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:19,411 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:19,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,411 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,411 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:19,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,411 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,411 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:19,411 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,411 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:19,411 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:19,411 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:19,411 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:19,411 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,411 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,412 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:19,412 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:19,412 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:19,412 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:19,412 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,412 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:19,412 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:19,412 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:19,412 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:19,412 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. started... 2023-07-24 12:15:19,413 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ed62c139047156182f9c4cdbe34edfa7 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 12:15:19,413 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,414 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,414 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,414 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:19,415 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,415 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:19,425 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/0724033c29594920bacf172a08ff3486 2023-07-24 12:15:19,437 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/193481541c9d48bfb07a11be0fa1f43b 2023-07-24 12:15:19,441 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/0724033c29594920bacf172a08ff3486 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/0724033c29594920bacf172a08ff3486 2023-07-24 12:15:19,445 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/0724033c29594920bacf172a08ff3486, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:15:19,445 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/193481541c9d48bfb07a11be0fa1f43b as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/193481541c9d48bfb07a11be0fa1f43b 2023-07-24 12:15:19,449 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/193481541c9d48bfb07a11be0fa1f43b, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:15:19,450 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ed62c139047156182f9c4cdbe34edfa7 in 37ms, sequenceid=6, compaction requested=false 2023-07-24 12:15:19,450 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:19,451 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ed62c139047156182f9c4cdbe34edfa7: 2023-07-24 12:15:19,451 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:19,451 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:19,451 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:19,451 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,451 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:19,451 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:19,453 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,453 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:19,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:19,453 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,453 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:19,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:19,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:19,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:19,455 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 12:15:19,456 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d5e31f6[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:19,456 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 12:15:19,456 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,457 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,457 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,457 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,457 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:19,457 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,457 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:19,457 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:19,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,457 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,457 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:19,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,457 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:19,457 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:19,457 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:19,457 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:19,458 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:19,457 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:19,463 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:19,463 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:19,463 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:19,464 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:19,464 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:19,464 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:19,464 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,464 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:19,464 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:19,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,464 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:19,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:19,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:19,464 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:19,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:19,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:19,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:15:19,467 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:15:19,467 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 12:15:19,467 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:19,467 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:19,467 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,467 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:19,469 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,469 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,469 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,469 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,469 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,469 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:19,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:19,469 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,469 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,469 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,470 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:19,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,470 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,471 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:19,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,472 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,472 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,472 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:19,473 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,473 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,485 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,485 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,485 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,485 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,485 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:19,485 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:19,485 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:19,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,485 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,485 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,485 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,485 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,485 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,486 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:19,486 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,486 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,486 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,486 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:19,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,486 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,486 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,486 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,486 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,486 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,486 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,486 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,486 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,487 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,586 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:19,587 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:19,600 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:19,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:19,602 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,602 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:19,602 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,602 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 12:15:19,602 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 12:15:19,602 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,602 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,603 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,603 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,603 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,603 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,604 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,604 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,604 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:19,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,604 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,605 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:19,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:19,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:19,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,605 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,605 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,605 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,605 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:19,605 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,605 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:19,605 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:19,606 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,605 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:19,606 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:19,606 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,606 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,606 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:19,606 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,606 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,606 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:19,606 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:19,606 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:19,606 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:19,606 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:19,606 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:19,606 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:19,606 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:19,606 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:19,606 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:19,606 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. started... 2023-07-24 12:15:19,607 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ed62c139047156182f9c4cdbe34edfa7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:19,608 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,608 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,608 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,608 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:19,609 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,609 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:19,617 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/4a1b0a2124fe4972a769ebadde402b77 2023-07-24 12:15:19,628 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/59e63b36c53b463c8e3f6fa553db2f4b 2023-07-24 12:15:19,633 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/4a1b0a2124fe4972a769ebadde402b77 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/4a1b0a2124fe4972a769ebadde402b77 2023-07-24 12:15:19,636 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/4a1b0a2124fe4972a769ebadde402b77, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:15:19,637 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/59e63b36c53b463c8e3f6fa553db2f4b as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/59e63b36c53b463c8e3f6fa553db2f4b 2023-07-24 12:15:19,641 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/59e63b36c53b463c8e3f6fa553db2f4b, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:15:19,642 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ed62c139047156182f9c4cdbe34edfa7 in 35ms, sequenceid=19, compaction requested=false 2023-07-24 12:15:19,642 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ed62c139047156182f9c4cdbe34edfa7: 2023-07-24 12:15:19,642 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:19,642 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:19,642 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:19,642 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,642 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:19,642 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:19,649 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,649 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:19,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:19,649 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,649 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:19,649 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:19,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:19,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,650 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,651 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:19,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 12:15:19,651 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@46ff4d76[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:19,651 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 12:15:19,651 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,652 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,652 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,652 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,652 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,652 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,653 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:19,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,653 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,653 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:19,653 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:19,653 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,653 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:19,653 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:19,653 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,653 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:19,653 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:19,653 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:19,653 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:19,653 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:19,654 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:19,654 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:19,654 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:19,654 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,654 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:19,654 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:19,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:19,654 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:19,655 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:19,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:19,655 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:19,655 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:19,655 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:19,655 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:19,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,656 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:19,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,657 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,658 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:15:19,658 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:15:19,658 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 12:15:19,658 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:19,658 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:19,658 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,658 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:19,661 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,661 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,661 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,661 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:19,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:19,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,661 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,661 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,661 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:19,661 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,661 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,661 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,662 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,662 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,662 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,662 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:19,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,663 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,664 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,664 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:19,665 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,665 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,665 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,667 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,668 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,668 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,668 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,668 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,668 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,668 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,668 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,668 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:19,668 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,668 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:19,668 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:19,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,668 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,668 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,668 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,668 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,668 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:19,669 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:19,668 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,669 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,669 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,669 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,669 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,669 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,669 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,669 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,769 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:19,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:19,784 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:19,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:19,786 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,786 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:19,786 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,786 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 12:15:19,786 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 12:15:19,786 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,786 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,788 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,788 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,788 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,788 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,788 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,788 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,788 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:19,788 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,789 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:19,789 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,789 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:19,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:19,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,790 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,790 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,790 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,790 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:19,790 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,790 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:19,790 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,790 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:19,790 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:19,790 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:19,792 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,792 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,792 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:19,795 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:19,795 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,795 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:19,795 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:19,795 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,795 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,795 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:19,795 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:19,795 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,795 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:19,795 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:19,795 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:19,795 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:19,795 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:19,795 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:19,795 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. started... 2023-07-24 12:15:19,795 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ed62c139047156182f9c4cdbe34edfa7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:19,798 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,798 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,798 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:19,807 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/facefb2574884ec39dca1d0bbc3febee 2023-07-24 12:15:19,820 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/b2370636c6504c23a0cf52ae01ffb6d4 2023-07-24 12:15:19,825 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/facefb2574884ec39dca1d0bbc3febee as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/facefb2574884ec39dca1d0bbc3febee 2023-07-24 12:15:19,829 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/facefb2574884ec39dca1d0bbc3febee, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:15:19,830 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/b2370636c6504c23a0cf52ae01ffb6d4 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/b2370636c6504c23a0cf52ae01ffb6d4 2023-07-24 12:15:19,834 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/b2370636c6504c23a0cf52ae01ffb6d4, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:15:19,836 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ed62c139047156182f9c4cdbe34edfa7 in 41ms, sequenceid=32, compaction requested=true 2023-07-24 12:15:19,836 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ed62c139047156182f9c4cdbe34edfa7: 2023-07-24 12:15:19,836 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:19,836 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:19,836 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:19,836 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,836 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:19,836 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:19,840 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,840 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:19,840 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:19,841 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,841 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:19,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:19,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:19,841 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,842 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,843 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:19,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 12:15:19,843 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4baf987d[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:19,843 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 12:15:19,843 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,844 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,844 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,844 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,844 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,844 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:19,844 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,844 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:19,845 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:19,844 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,845 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:19,845 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:19,845 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:19,845 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:19,844 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:19,845 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:19,845 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:19,850 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:19,850 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:19,850 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:19,851 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:19,851 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:19,851 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:19,851 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,851 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:19,851 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:19,851 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:19,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:19,851 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:19,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:19,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:19,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:19,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,855 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:15:19,855 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:15:19,855 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 12:15:19,855 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:19,855 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:19,855 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,855 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:19,856 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,856 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,856 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,856 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:19,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:19,856 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,856 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:19,856 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,856 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,857 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,857 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,857 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,858 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,858 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:19,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,858 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,859 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:19,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,860 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,861 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,861 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,861 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,863 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,863 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,863 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,863 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,863 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,863 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,863 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:19,864 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:19,863 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,863 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:19,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,864 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:19,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,864 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:19,864 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:19,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,864 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:19,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:19,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,865 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:19,865 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:19,964 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:19,965 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:19,978 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:19,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:19,979 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,980 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:19,980 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,980 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 12:15:19,980 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 12:15:19,980 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,980 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,982 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,982 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,982 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,982 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:19,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,982 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:19,982 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,983 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:19,983 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:19,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:19,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,983 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,983 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,983 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,984 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:19,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,984 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:19,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,984 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,984 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:19,984 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,984 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:19,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:19,984 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,984 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,984 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,985 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:19,985 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:19,985 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:19,986 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,986 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,986 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:19,987 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:19,987 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,987 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:19,987 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:19,987 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,987 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,987 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:19,987 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:19,987 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:19,987 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:19,988 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:19,988 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:19,988 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:19,988 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:19,988 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:19,988 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. started... 2023-07-24 12:15:19,988 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ed62c139047156182f9c4cdbe34edfa7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:19,989 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,989 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:19,989 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:19,998 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/934c8cda31d2429abcd32850302edc0b 2023-07-24 12:15:20,012 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/1dff5c81a99942fa878626401a45997e 2023-07-24 12:15:20,017 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/934c8cda31d2429abcd32850302edc0b as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/934c8cda31d2429abcd32850302edc0b 2023-07-24 12:15:20,021 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/934c8cda31d2429abcd32850302edc0b, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:15:20,022 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/1dff5c81a99942fa878626401a45997e as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/1dff5c81a99942fa878626401a45997e 2023-07-24 12:15:20,026 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/1dff5c81a99942fa878626401a45997e, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:15:20,027 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ed62c139047156182f9c4cdbe34edfa7 in 39ms, sequenceid=45, compaction requested=true 2023-07-24 12:15:20,027 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ed62c139047156182f9c4cdbe34edfa7: 2023-07-24 12:15:20,027 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:20,027 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:20,027 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:20,027 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,027 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,027 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,029 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,029 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,029 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,029 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,029 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,030 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,031 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,031 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 12:15:20,031 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4d7265d6[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:20,031 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 12:15:20,031 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,033 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,033 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,033 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,033 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,033 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,033 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:20,033 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,033 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,033 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,033 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,033 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:20,033 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,033 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:20,033 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,034 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,034 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:20,036 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,036 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,036 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,037 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,037 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,037 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,037 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,037 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,037 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,037 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,038 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,039 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,040 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:15:20,040 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:15:20,040 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 12:15:20,040 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:20,040 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:20,040 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,040 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:20,045 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,045 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,045 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,045 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,045 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,045 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,045 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,046 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,045 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,046 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,046 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,046 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,046 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,047 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,047 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,048 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,049 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,049 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,049 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,049 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,050 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,052 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,052 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,052 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,052 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,052 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:20,052 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:20,052 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,052 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,052 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,052 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:20,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,052 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,052 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,053 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:20,053 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,053 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,053 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,053 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,053 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,053 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,053 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,053 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,053 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,153 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:20,154 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:20,166 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:20,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:20,168 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,168 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:20,168 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,168 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 12:15:20,168 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 12:15:20,168 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,168 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,170 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,170 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,170 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,170 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,170 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,170 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,170 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:20,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,171 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,171 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,171 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,171 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,171 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,171 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,171 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,171 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,171 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,171 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,171 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,171 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,171 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,172 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,172 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,172 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,171 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,172 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,172 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,172 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,172 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,172 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,172 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,172 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,172 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,172 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:20,172 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:20,172 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:20,172 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. started... 2023-07-24 12:15:20,172 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ed62c139047156182f9c4cdbe34edfa7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:20,179 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,179 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,180 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,180 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,180 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,180 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,182 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/f068438ae0bf4f42b1e48c57a80e0a04 2023-07-24 12:15:20,193 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/0db122c7c0f84a56b5c228994731bbb0 2023-07-24 12:15:20,198 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/f068438ae0bf4f42b1e48c57a80e0a04 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/f068438ae0bf4f42b1e48c57a80e0a04 2023-07-24 12:15:20,202 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/f068438ae0bf4f42b1e48c57a80e0a04, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:15:20,202 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/0db122c7c0f84a56b5c228994731bbb0 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/0db122c7c0f84a56b5c228994731bbb0 2023-07-24 12:15:20,206 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/0db122c7c0f84a56b5c228994731bbb0, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:15:20,207 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ed62c139047156182f9c4cdbe34edfa7 in 35ms, sequenceid=58, compaction requested=true 2023-07-24 12:15:20,207 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ed62c139047156182f9c4cdbe34edfa7: 2023-07-24 12:15:20,207 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:20,207 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:20,207 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:20,207 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,207 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,207 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,210 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,210 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,210 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,210 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,210 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,211 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,212 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 12:15:20,213 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@51bf3c0f[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:20,213 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 12:15:20,213 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,214 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,214 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,214 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,214 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,214 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,214 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,214 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,214 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,214 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:20,214 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,214 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,215 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:20,214 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:20,215 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,215 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,215 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:20,216 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,216 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,216 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,217 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,217 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,217 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,219 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,219 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,219 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,219 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:15:20,222 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:15:20,222 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 12:15:20,222 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:20,222 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:20,222 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,222 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:20,224 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,224 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,224 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,224 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,225 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,224 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,224 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,224 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,225 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,225 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,225 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,226 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,226 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,227 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,228 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,229 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,229 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,229 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,232 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,232 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,232 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,232 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,232 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,232 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,232 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,232 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,232 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,232 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,232 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,232 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,233 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,233 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,233 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,233 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,232 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:20,233 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:20,232 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,233 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,233 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,233 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,233 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,233 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,233 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:20,233 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:20,234 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,234 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,334 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:20,335 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:20,347 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:20,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:20,348 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,349 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:20,349 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,349 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 12:15:20,349 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 12:15:20,349 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,349 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,354 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,354 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,354 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,354 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,354 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,354 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:20,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,355 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,355 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,355 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,355 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,355 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,355 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,356 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,356 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,356 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,356 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,356 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,356 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:20,356 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:20,356 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:20,356 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. started... 2023-07-24 12:15:20,356 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ed62c139047156182f9c4cdbe34edfa7 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:15:20,357 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,358 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,358 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,358 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,358 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,358 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,366 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/3c849680a43947559da3bae87e1ed514 2023-07-24 12:15:20,379 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/ec16a8641a10453596dedcf56d2c6d25 2023-07-24 12:15:20,383 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/3c849680a43947559da3bae87e1ed514 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/3c849680a43947559da3bae87e1ed514 2023-07-24 12:15:20,386 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/3c849680a43947559da3bae87e1ed514, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:15:20,387 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/ec16a8641a10453596dedcf56d2c6d25 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/ec16a8641a10453596dedcf56d2c6d25 2023-07-24 12:15:20,391 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/ec16a8641a10453596dedcf56d2c6d25, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:15:20,391 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ed62c139047156182f9c4cdbe34edfa7 in 35ms, sequenceid=71, compaction requested=true 2023-07-24 12:15:20,391 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ed62c139047156182f9c4cdbe34edfa7: 2023-07-24 12:15:20,392 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:20,392 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:20,392 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:20,392 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,392 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,392 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,393 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,393 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,393 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,394 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,394 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,394 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,395 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 12:15:20,396 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6d975fb3[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:20,396 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 12:15:20,396 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,397 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,397 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,397 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,397 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,397 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,397 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,397 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,397 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,397 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:20,397 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,397 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:20,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,397 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:20,397 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,397 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,397 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:20,400 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,400 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,400 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,400 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,400 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,400 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,400 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,400 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,400 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,401 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,401 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,401 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,402 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,403 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:15:20,403 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:15:20,403 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 12:15:20,403 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:20,403 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:20,403 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,403 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:20,405 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,405 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,405 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,405 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,405 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,405 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,405 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,405 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,405 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,406 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,406 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,408 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,408 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,408 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,408 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,409 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,411 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,411 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,411 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,411 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,411 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,411 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,411 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,411 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,411 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,411 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,411 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,411 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,411 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,411 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:20,412 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:20,411 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,411 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,411 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,412 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,412 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,412 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,412 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,412 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:20,412 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:20,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,412 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,412 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,512 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:20,513 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:20,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:20,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:20,528 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,528 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:20,528 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,528 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 12:15:20,528 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 12:15:20,528 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,528 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,530 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,530 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,531 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,531 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,531 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,531 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,531 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:20,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,531 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,531 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,532 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,532 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,532 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,532 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,532 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,532 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,532 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,532 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,532 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,532 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,532 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,533 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,533 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,533 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,533 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,533 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,533 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,533 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,533 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,533 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:20,533 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:20,533 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:20,533 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. started... 2023-07-24 12:15:20,533 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,533 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,533 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ed62c139047156182f9c4cdbe34edfa7 2/2 column families, dataSize=530 B heapSize=2.33 KB 2023-07-24 12:15:20,533 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,533 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,533 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,534 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,535 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,535 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,535 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,535 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,535 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,543 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7 2023-07-24 12:15:20,548 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b2f3ae3cde941b5a357a8f4f857d1e7 2023-07-24 12:15:20,556 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=265 B at sequenceid=83 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/2ba017bbab1e409087169dd4001ce0ce 2023-07-24 12:15:20,559 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ba017bbab1e409087169dd4001ce0ce 2023-07-24 12:15:20,560 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7 2023-07-24 12:15:20,564 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b2f3ae3cde941b5a357a8f4f857d1e7 2023-07-24 12:15:20,564 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7, entries=9, sequenceid=83, filesize=5.1 K 2023-07-24 12:15:20,565 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/2ba017bbab1e409087169dd4001ce0ce as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/2ba017bbab1e409087169dd4001ce0ce 2023-07-24 12:15:20,568 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ba017bbab1e409087169dd4001ce0ce 2023-07-24 12:15:20,568 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/2ba017bbab1e409087169dd4001ce0ce, entries=9, sequenceid=83, filesize=5.1 K 2023-07-24 12:15:20,569 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~530 B/530, heapSize ~2.30 KB/2352, currentSize=0 B/0 for ed62c139047156182f9c4cdbe34edfa7 in 36ms, sequenceid=83, compaction requested=true 2023-07-24 12:15:20,569 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ed62c139047156182f9c4cdbe34edfa7: 2023-07-24 12:15:20,569 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:20,569 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:20,569 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:20,569 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,569 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,569 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,571 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,571 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,571 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,571 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,573 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,573 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 12:15:20,573 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5a4c5e0c[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:20,573 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 12:15:20,573 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,579 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,579 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,579 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,579 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,580 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,580 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,580 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,579 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,580 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:20,579 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,580 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,580 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:20,580 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,580 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:20,580 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:20,583 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,583 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,583 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,583 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,583 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,584 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,584 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,584 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,584 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,584 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,584 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,584 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,585 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,586 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:15:20,587 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:15:20,587 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 12:15:20,587 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:20,587 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:20,587 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,587 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:20,590 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,590 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,590 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,590 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,590 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,590 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,590 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,590 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,590 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,591 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,591 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,591 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,591 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,592 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,592 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,593 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,593 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,593 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,594 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,594 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,596 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,596 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,596 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,596 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,596 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,596 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,596 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,596 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,596 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:20,597 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:20,596 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,596 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,597 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,596 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,596 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,597 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:20,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,597 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:20,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,597 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,597 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,597 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,597 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,597 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,597 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,597 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,597 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,597 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,598 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,697 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:20,698 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:20,706 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:20,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:20,707 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,707 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:20,707 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,707 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 12:15:20,707 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 12:15:20,708 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,708 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,709 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,709 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,709 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,709 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,709 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,709 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,709 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:20,709 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,710 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,710 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,710 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,710 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,711 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,710 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,711 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,711 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,711 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,711 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,711 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,711 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,711 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:20,711 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:20,711 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:20,711 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. started... 2023-07-24 12:15:20,712 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ed62c139047156182f9c4cdbe34edfa7 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-07-24 12:15:20,713 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,713 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,713 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,713 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,713 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,713 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,721 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/aede97c2b22f49a4b7797be56223204a 2023-07-24 12:15:20,725 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aede97c2b22f49a4b7797be56223204a 2023-07-24 12:15:20,734 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=91 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/334c70b508ad47909996d05ffb25115e 2023-07-24 12:15:20,738 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 334c70b508ad47909996d05ffb25115e 2023-07-24 12:15:20,738 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/aede97c2b22f49a4b7797be56223204a as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/aede97c2b22f49a4b7797be56223204a 2023-07-24 12:15:20,742 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aede97c2b22f49a4b7797be56223204a 2023-07-24 12:15:20,742 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/aede97c2b22f49a4b7797be56223204a, entries=5, sequenceid=91, filesize=5.0 K 2023-07-24 12:15:20,743 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/334c70b508ad47909996d05ffb25115e as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/334c70b508ad47909996d05ffb25115e 2023-07-24 12:15:20,747 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 334c70b508ad47909996d05ffb25115e 2023-07-24 12:15:20,747 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/334c70b508ad47909996d05ffb25115e, entries=5, sequenceid=91, filesize=5.0 K 2023-07-24 12:15:20,748 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for ed62c139047156182f9c4cdbe34edfa7 in 36ms, sequenceid=91, compaction requested=true 2023-07-24 12:15:20,748 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ed62c139047156182f9c4cdbe34edfa7: 2023-07-24 12:15:20,748 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:20,748 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:20,748 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:20,748 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,748 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,748 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,755 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,755 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,755 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,755 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 12:15:20,757 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4cedc46[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:20,757 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 12:15:20,757 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,758 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,758 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,758 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,758 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,759 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,759 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,759 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,758 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,759 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:20,759 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,759 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:20,759 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,759 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,759 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:20,759 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:20,760 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,760 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,760 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,761 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,761 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,761 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,761 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,761 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,761 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,761 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,761 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,762 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:15:20,764 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:15:20,764 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 12:15:20,764 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:20,765 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:20,765 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,765 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:20,767 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,767 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,767 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,767 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,767 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,767 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,767 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,767 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,767 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,767 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,768 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,768 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,768 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,768 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,769 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,770 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,770 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,770 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,770 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,771 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,775 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,776 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,776 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,776 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,776 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,776 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:20,776 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:20,776 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,776 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,776 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,776 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,776 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,776 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:20,776 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,776 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,776 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,776 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:20,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,777 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,777 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,777 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,776 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,777 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,777 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,777 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,777 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,877 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:20,877 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:20,884 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:15:20,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionDeletedRowsDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:15:20,886 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,886 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:20,886 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,886 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' 2023-07-24 12:15:20,886 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionDeletedRowsDaughter', kicking off acquire phase on members. 2023-07-24 12:15:20,886 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,886 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,892 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,892 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,892 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,892 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,892 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,892 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,893 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,893 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:15:20,893 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,893 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,893 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,894 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,894 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:15:20,894 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,894 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,894 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,894 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,894 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,894 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,894 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,894 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionDeletedRowsDaughter is still present, removing 2023-07-24 12:15:20,894 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,894 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,894 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,894 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,894 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,894 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionDeletedRowsDaughter' with timeout 60000ms 2023-07-24 12:15:20,894 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,894 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:15:20,894 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:15:20,894 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:15:20,894 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' starting 'acquire' stage 2023-07-24 12:15:20,894 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,894 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:20,895 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,895 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. started... 2023-07-24 12:15:20,894 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,895 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,895 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ed62c139047156182f9c4cdbe34edfa7 2/2 column families, dataSize=250 B heapSize=1.52 KB 2023-07-24 12:15:20,896 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,897 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,897 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,897 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,897 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,897 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,904 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/8b6c10a14d1d42b2a42745390bcbe281 2023-07-24 12:15:20,908 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8b6c10a14d1d42b2a42745390bcbe281 2023-07-24 12:15:20,920 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=125 B at sequenceid=99 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/0556280e1dd14b4ba397a8cc48d0663e 2023-07-24 12:15:20,923 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0556280e1dd14b4ba397a8cc48d0663e 2023-07-24 12:15:20,925 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/8b6c10a14d1d42b2a42745390bcbe281 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/8b6c10a14d1d42b2a42745390bcbe281 2023-07-24 12:15:20,929 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8b6c10a14d1d42b2a42745390bcbe281 2023-07-24 12:15:20,929 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/8b6c10a14d1d42b2a42745390bcbe281, entries=5, sequenceid=99, filesize=5.0 K 2023-07-24 12:15:20,930 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/0556280e1dd14b4ba397a8cc48d0663e as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/0556280e1dd14b4ba397a8cc48d0663e 2023-07-24 12:15:20,933 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0556280e1dd14b4ba397a8cc48d0663e 2023-07-24 12:15:20,934 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/0556280e1dd14b4ba397a8cc48d0663e, entries=5, sequenceid=99, filesize=5.0 K 2023-07-24 12:15:20,934 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~250 B/250, heapSize ~1.48 KB/1520, currentSize=0 B/0 for ed62c139047156182f9c4cdbe34edfa7 in 39ms, sequenceid=99, compaction requested=true 2023-07-24 12:15:20,934 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ed62c139047156182f9c4cdbe34edfa7: 2023-07-24 12:15:20,934 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:20,935 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:15:20,935 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:15:20,935 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,935 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally acquired 2023-07-24 12:15:20,935 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionDeletedRowsDaughter) in zk 2023-07-24 12:15:20,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,936 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,937 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,937 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:15:20,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,938 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionDeletedRowsDaughter' on coordinator 2023-07-24 12:15:20,939 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2c39d794[Count = 0] remaining members to acquire global barrier 2023-07-24 12:15:20,939 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionDeletedRowsDaughter' starting 'in-barrier' execution. 2023-07-24 12:15:20,939 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,940 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,940 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,940 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,940 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,940 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,940 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,940 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:15:20,940 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,940 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:15:20,940 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,940 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,940 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,941 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:15:20,940 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' received 'reached' from coordinator. 2023-07-24 12:15:20,941 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' locally completed 2023-07-24 12:15:20,941 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionDeletedRowsDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:15:20,942 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,942 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,942 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,942 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,942 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,943 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,943 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' has notified controller of completion 2023-07-24 12:15:20,943 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,943 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,943 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,943 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,943 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionDeletedRowsDaughter' completed. 2023-07-24 12:15:20,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,944 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,945 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,946 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionDeletedRowsDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:15:20,946 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionDeletedRowsDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:15:20,946 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionDeletedRowsDaughter' execution completed 2023-07-24 12:15:20,946 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:15:20,946 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:15:20,946 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,946 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionDeletedRowsDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:15:20,948 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,948 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,948 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,948 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:15:20,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:15:20,948 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,948 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,948 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,948 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,949 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,949 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:15:20,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,949 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,949 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,949 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,950 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:15:20,950 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,951 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,951 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:15:20,952 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,952 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,952 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,954 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,954 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,954 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,954 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,954 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,954 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,954 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,954 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,955 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionDeletedRowsDaughter' 2023-07-24 12:15:20,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,954 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:15:20,955 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:15:20,954 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:15:20,954 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,954 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:20,954 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,955 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:15:20,955 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionDeletedRowsDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:15:20,955 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:15:20,955 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:15:20,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,955 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:15:20,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:15:20,955 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:20,955 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,955 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:20,955 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:20,956 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:15:20,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:15:21,055 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:15:21,056 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:15:21,059 DEBUG [Listener at localhost/37279] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:21,059 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:21,061 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:21,061 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:21,061 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=72, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:21,062 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ed62c139047156182f9c4cdbe34edfa7, UNASSIGN}] 2023-07-24 12:15:21,063 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=73, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ed62c139047156182f9c4cdbe34edfa7, UNASSIGN 2023-07-24 12:15:21,063 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=ed62c139047156182f9c4cdbe34edfa7, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:21,064 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1690200921063"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200921063"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200921063"}]},"ts":"1690200921063"} 2023-07-24 12:15:21,065 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=74, ppid=73, state=RUNNABLE; CloseRegionProcedure ed62c139047156182f9c4cdbe34edfa7, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:15:21,218 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing ed62c139047156182f9c4cdbe34edfa7, disabling compactions & flushes 2023-07-24 12:15:21,218 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:21,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:21,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. after waiting 0 ms 2023-07-24 12:15:21,218 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:21,218 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2745): Flushing ed62c139047156182f9c4cdbe34edfa7 2/2 column families, dataSize=50 B heapSize=720 B 2023-07-24 12:15:21,228 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/dc24e7440e4849ffb253557dbbfc9577 2023-07-24 12:15:21,234 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc24e7440e4849ffb253557dbbfc9577 2023-07-24 12:15:21,243 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=25 B at sequenceid=103 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/1af444bcfe724d6485ba240a2c869098 2023-07-24 12:15:21,249 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1af444bcfe724d6485ba240a2c869098 2023-07-24 12:15:21,250 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf1/dc24e7440e4849ffb253557dbbfc9577 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/dc24e7440e4849ffb253557dbbfc9577 2023-07-24 12:15:21,253 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc24e7440e4849ffb253557dbbfc9577 2023-07-24 12:15:21,254 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/dc24e7440e4849ffb253557dbbfc9577, entries=1, sequenceid=103, filesize=4.9 K 2023-07-24 12:15:21,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.tmp/cf2/1af444bcfe724d6485ba240a2c869098 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/1af444bcfe724d6485ba240a2c869098 2023-07-24 12:15:21,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1af444bcfe724d6485ba240a2c869098 2023-07-24 12:15:21,258 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/1af444bcfe724d6485ba240a2c869098, entries=1, sequenceid=103, filesize=4.9 K 2023-07-24 12:15:21,259 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~50 B/50, heapSize ~688 B/688, currentSize=0 B/0 for ed62c139047156182f9c4cdbe34edfa7 in 41ms, sequenceid=103, compaction requested=true 2023-07-24 12:15:21,264 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=1 2023-07-24 12:15:21,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:21,265 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7. 2023-07-24 12:15:21,265 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for ed62c139047156182f9c4cdbe34edfa7: 2023-07-24 12:15:21,266 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,267 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=73 updating hbase:meta row=ed62c139047156182f9c4cdbe34edfa7, regionState=CLOSED 2023-07-24 12:15:21,267 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1690200921267"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200921267"}]},"ts":"1690200921267"} 2023-07-24 12:15:21,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=74, resume processing ppid=73 2023-07-24 12:15:21,270 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=74, ppid=73, state=SUCCESS; CloseRegionProcedure ed62c139047156182f9c4cdbe34edfa7, server=jenkins-hbase6.apache.org,34575,1690200894532 in 203 msec 2023-07-24 12:15:21,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=73, resume processing ppid=72 2023-07-24 12:15:21,272 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=73, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=ed62c139047156182f9c4cdbe34edfa7, UNASSIGN in 208 msec 2023-07-24 12:15:21,288 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=72 splitting 20 storefiles, region=ed62c139047156182f9c4cdbe34edfa7, threads=16 2023-07-24 12:15:21,288 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/0556280e1dd14b4ba397a8cc48d0663e for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,288 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/0db122c7c0f84a56b5c228994731bbb0 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,291 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/193481541c9d48bfb07a11be0fa1f43b for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,292 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/1af444bcfe724d6485ba240a2c869098 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,292 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/1dff5c81a99942fa878626401a45997e for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,292 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/2ba017bbab1e409087169dd4001ce0ce for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,295 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/334c70b508ad47909996d05ffb25115e for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,299 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/59e63b36c53b463c8e3f6fa553db2f4b for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,299 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/b2370636c6504c23a0cf52ae01ffb6d4 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,302 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/ec16a8641a10453596dedcf56d2c6d25 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,302 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/0724033c29594920bacf172a08ff3486 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,304 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0556280e1dd14b4ba397a8cc48d0663e 2023-07-24 12:15:21,304 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/0db122c7c0f84a56b5c228994731bbb0, top=false 2023-07-24 12:15:21,304 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1af444bcfe724d6485ba240a2c869098 2023-07-24 12:15:21,307 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/3c849680a43947559da3bae87e1ed514 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,307 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/4a1b0a2124fe4972a769ebadde402b77 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,315 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,315 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/8b6c10a14d1d42b2a42745390bcbe281 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,316 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/934c8cda31d2429abcd32850302edc0b for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,318 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/1dff5c81a99942fa878626401a45997e, top=false 2023-07-24 12:15:21,323 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0556280e1dd14b4ba397a8cc48d0663e 2023-07-24 12:15:21,323 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/0556280e1dd14b4ba397a8cc48d0663e, top=true 2023-07-24 12:15:21,325 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ba017bbab1e409087169dd4001ce0ce 2023-07-24 12:15:21,326 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1af444bcfe724d6485ba240a2c869098 2023-07-24 12:15:21,326 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/1af444bcfe724d6485ba240a2c869098, top=true 2023-07-24 12:15:21,326 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 334c70b508ad47909996d05ffb25115e 2023-07-24 12:15:21,327 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/ec16a8641a10453596dedcf56d2c6d25, top=false 2023-07-24 12:15:21,328 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8b6c10a14d1d42b2a42745390bcbe281 2023-07-24 12:15:21,329 DEBUG [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/934c8cda31d2429abcd32850302edc0b, top=false 2023-07-24 12:15:21,331 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/3c849680a43947559da3bae87e1ed514, top=false 2023-07-24 12:15:21,333 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b2f3ae3cde941b5a357a8f4f857d1e7 2023-07-24 12:15:21,338 INFO [StoreFileSplitter-pool-14] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8b6c10a14d1d42b2a42745390bcbe281 2023-07-24 12:15:21,338 DEBUG [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/8b6c10a14d1d42b2a42745390bcbe281, top=true 2023-07-24 12:15:21,340 INFO [StoreFileSplitter-pool-6] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 334c70b508ad47909996d05ffb25115e 2023-07-24 12:15:21,340 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/334c70b508ad47909996d05ffb25115e, top=true 2023-07-24 12:15:21,369 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.splits/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-0db122c7c0f84a56b5c228994731bbb0 for child: d6d1a02c2c1c2259a534e0cfc7642c20, parent: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,369 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.splits/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-ec16a8641a10453596dedcf56d2c6d25 for child: d6d1a02c2c1c2259a534e0cfc7642c20, parent: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,369 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.splits/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-1af444bcfe724d6485ba240a2c869098 for child: 7d24409abbbbe6199617e8fdd3054481, parent: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,369 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/1af444bcfe724d6485ba240a2c869098 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,369 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/aede97c2b22f49a4b7797be56223204a for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,369 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.splits/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-3c849680a43947559da3bae87e1ed514 for child: d6d1a02c2c1c2259a534e0cfc7642c20, parent: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,369 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.splits/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-0556280e1dd14b4ba397a8cc48d0663e for child: 7d24409abbbbe6199617e8fdd3054481, parent: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,369 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/0556280e1dd14b4ba397a8cc48d0663e for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,369 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/dc24e7440e4849ffb253557dbbfc9577 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,373 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aede97c2b22f49a4b7797be56223204a 2023-07-24 12:15:21,376 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/3c849680a43947559da3bae87e1ed514 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,376 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/f068438ae0bf4f42b1e48c57a80e0a04 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,376 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/ec16a8641a10453596dedcf56d2c6d25 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,376 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=72 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/facefb2574884ec39dca1d0bbc3febee for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,377 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc24e7440e4849ffb253557dbbfc9577 2023-07-24 12:15:21,379 INFO [StoreFileSplitter-pool-3] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aede97c2b22f49a4b7797be56223204a 2023-07-24 12:15:21,379 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/aede97c2b22f49a4b7797be56223204a, top=true 2023-07-24 12:15:21,380 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/0db122c7c0f84a56b5c228994731bbb0 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,381 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/f068438ae0bf4f42b1e48c57a80e0a04, top=false 2023-07-24 12:15:21,383 INFO [StoreFileSplitter-pool-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc24e7440e4849ffb253557dbbfc9577 2023-07-24 12:15:21,383 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(700): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/dc24e7440e4849ffb253557dbbfc9577, top=true 2023-07-24 12:15:21,392 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.splits/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-1dff5c81a99942fa878626401a45997e for child: d6d1a02c2c1c2259a534e0cfc7642c20, parent: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,392 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.splits/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-334c70b508ad47909996d05ffb25115e for child: 7d24409abbbbe6199617e8fdd3054481, parent: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,392 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/334c70b508ad47909996d05ffb25115e for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,392 INFO [StoreFileSplitter-pool-15] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.splits/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-934c8cda31d2429abcd32850302edc0b for child: d6d1a02c2c1c2259a534e0cfc7642c20, parent: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,393 INFO [StoreFileSplitter-pool-14] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.splits/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-8b6c10a14d1d42b2a42745390bcbe281 for child: 7d24409abbbbe6199617e8fdd3054481, parent: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,394 DEBUG [StoreFileSplitter-pool-14] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/8b6c10a14d1d42b2a42745390bcbe281 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,397 DEBUG [StoreFileSplitter-pool-15] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/934c8cda31d2429abcd32850302edc0b for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,397 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/1dff5c81a99942fa878626401a45997e for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,423 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.splits/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-dc24e7440e4849ffb253557dbbfc9577 for child: 7d24409abbbbe6199617e8fdd3054481, parent: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,424 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/dc24e7440e4849ffb253557dbbfc9577 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,424 INFO [StoreFileSplitter-pool-5] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ba017bbab1e409087169dd4001ce0ce 2023-07-24 12:15:21,431 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.splits/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-f068438ae0bf4f42b1e48c57a80e0a04 for child: d6d1a02c2c1c2259a534e0cfc7642c20, parent: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,439 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/.splits/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-aede97c2b22f49a4b7797be56223204a for child: 7d24409abbbbe6199617e8fdd3054481, parent: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,439 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/aede97c2b22f49a4b7797be56223204a for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,443 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/f068438ae0bf4f42b1e48c57a80e0a04 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,457 INFO [StoreFileSplitter-pool-13] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b2f3ae3cde941b5a357a8f4f857d1e7 2023-07-24 12:15:21,499 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/2ba017bbab1e409087169dd4001ce0ce for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,500 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/b2370636c6504c23a0cf52ae01ffb6d4 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,502 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/193481541c9d48bfb07a11be0fa1f43b for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,511 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/0724033c29594920bacf172a08ff3486 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,515 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/59e63b36c53b463c8e3f6fa553db2f4b for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,516 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,525 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/4a1b0a2124fe4972a769ebadde402b77 for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,526 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=72 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/facefb2574884ec39dca1d0bbc3febee for region: ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,526 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=72 split storefiles for region ed62c139047156182f9c4cdbe34edfa7 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 12:15:21,574 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-07-24 12:15:21,576 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/recovered.edits/106.seqid, newMaxSeqId=106, maxSeqId=-1 2023-07-24 12:15:21,577 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7.","families":{"info":[{"qualifier":"regioninfo","vlen":73,"tag":[],"timestamp":"1690200921577"},{"qualifier":"splitA","vlen":75,"tag":[],"timestamp":"1690200921577"},{"qualifier":"splitB","vlen":75,"tag":[],"timestamp":"1690200921577"}]},"ts":"1690200921577"} 2023-07-24 12:15:21,577 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690200921577"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200921577"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200921577"}]},"ts":"1690200921577"} 2023-07-24 12:15:21,577 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690200921577"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200921577"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200921577"}]},"ts":"1690200921577"} 2023-07-24 12:15:21,580 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d6d1a02c2c1c2259a534e0cfc7642c20, ASSIGN}, {pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7d24409abbbbe6199617e8fdd3054481, ASSIGN}] 2023-07-24 12:15:21,581 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7d24409abbbbe6199617e8fdd3054481, ASSIGN 2023-07-24 12:15:21,581 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=76, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7d24409abbbbe6199617e8fdd3054481, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase6.apache.org,34575,1690200894532; forceNewPlan=false, retain=false 2023-07-24 12:15:21,582 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d6d1a02c2c1c2259a534e0cfc7642c20, ASSIGN 2023-07-24 12:15:21,582 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=75, ppid=72, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d6d1a02c2c1c2259a534e0cfc7642c20, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase6.apache.org,34575,1690200894532; forceNewPlan=false, retain=false 2023-07-24 12:15:21,732 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-24 12:15:21,733 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=7d24409abbbbe6199617e8fdd3054481, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:21,733 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690200921733"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200921733"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200921733"}]},"ts":"1690200921733"} 2023-07-24 12:15:21,735 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=77, ppid=76, state=RUNNABLE; OpenRegionProcedure 7d24409abbbbe6199617e8fdd3054481, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:15:21,737 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=d6d1a02c2c1c2259a534e0cfc7642c20, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:21,737 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690200921737"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200921737"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200921737"}]},"ts":"1690200921737"} 2023-07-24 12:15:21,738 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=78, ppid=75, state=RUNNABLE; OpenRegionProcedure d6d1a02c2c1c2259a534e0cfc7642c20, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:15:21,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. 2023-07-24 12:15:21,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 7d24409abbbbe6199617e8fdd3054481, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.', STARTKEY => '60', ENDKEY => ''} 2023-07-24 12:15:21,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:21,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter 7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:21,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:21,890 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:21,891 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:21,892 INFO [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:21,893 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1 2023-07-24 12:15:21,893 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1 2023-07-24 12:15:21,893 INFO [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7d24409abbbbe6199617e8fdd3054481 columnFamilyName cf1 2023-07-24 12:15:21,904 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/0724033c29594920bacf172a08ff3486.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/0724033c29594920bacf172a08ff3486-top 2023-07-24 12:15:21,909 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/4a1b0a2124fe4972a769ebadde402b77.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/4a1b0a2124fe4972a769ebadde402b77-top 2023-07-24 12:15:21,912 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b2f3ae3cde941b5a357a8f4f857d1e7.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,913 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7-top 2023-07-24 12:15:21,917 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/facefb2574884ec39dca1d0bbc3febee.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/facefb2574884ec39dca1d0bbc3febee-top 2023-07-24 12:15:21,920 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 8b6c10a14d1d42b2a42745390bcbe281 2023-07-24 12:15:21,921 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-8b6c10a14d1d42b2a42745390bcbe281 2023-07-24 12:15:21,924 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for aede97c2b22f49a4b7797be56223204a 2023-07-24 12:15:21,924 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-aede97c2b22f49a4b7797be56223204a 2023-07-24 12:15:21,928 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for dc24e7440e4849ffb253557dbbfc9577 2023-07-24 12:15:21,928 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-dc24e7440e4849ffb253557dbbfc9577 2023-07-24 12:15:21,928 INFO [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(310): Store=7d24409abbbbe6199617e8fdd3054481/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:21,928 INFO [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:21,929 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2 2023-07-24 12:15:21,929 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2 2023-07-24 12:15:21,929 INFO [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 7d24409abbbbe6199617e8fdd3054481 columnFamilyName cf2 2023-07-24 12:15:21,940 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/193481541c9d48bfb07a11be0fa1f43b.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/193481541c9d48bfb07a11be0fa1f43b-top 2023-07-24 12:15:21,943 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ba017bbab1e409087169dd4001ce0ce.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:21,944 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/2ba017bbab1e409087169dd4001ce0ce.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/2ba017bbab1e409087169dd4001ce0ce-top 2023-07-24 12:15:21,948 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/59e63b36c53b463c8e3f6fa553db2f4b.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/59e63b36c53b463c8e3f6fa553db2f4b-top 2023-07-24 12:15:21,952 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/b2370636c6504c23a0cf52ae01ffb6d4.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/b2370636c6504c23a0cf52ae01ffb6d4-top 2023-07-24 12:15:21,955 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 0556280e1dd14b4ba397a8cc48d0663e 2023-07-24 12:15:21,955 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-0556280e1dd14b4ba397a8cc48d0663e 2023-07-24 12:15:21,958 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1af444bcfe724d6485ba240a2c869098 2023-07-24 12:15:21,958 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-1af444bcfe724d6485ba240a2c869098 2023-07-24 12:15:21,962 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 334c70b508ad47909996d05ffb25115e 2023-07-24 12:15:21,962 DEBUG [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-334c70b508ad47909996d05ffb25115e 2023-07-24 12:15:21,962 INFO [StoreOpener-7d24409abbbbe6199617e8fdd3054481-1] regionserver.HStore(310): Store=7d24409abbbbe6199617e8fdd3054481/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:21,963 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:21,964 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:21,966 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:15:21,967 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:21,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 7d24409abbbbe6199617e8fdd3054481; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72377637, jitterRate=0.0785108357667923}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:15:21,968 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 7d24409abbbbe6199617e8fdd3054481: 2023-07-24 12:15:21,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481., pid=77, masterSystemTime=1690200921886 2023-07-24 12:15:21,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:21,969 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 12:15:21,971 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:15:21,971 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:15:21,971 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. 2023-07-24 12:15:21,971 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1912): 7d24409abbbbe6199617e8fdd3054481/cf1 is initiating minor compaction (all files) 2023-07-24 12:15:21,971 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. 2023-07-24 12:15:21,971 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1912): 7d24409abbbbe6199617e8fdd3054481/cf2 is initiating minor compaction (all files) 2023-07-24 12:15:21,971 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 7d24409abbbbe6199617e8fdd3054481/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. 2023-07-24 12:15:21,971 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 7d24409abbbbe6199617e8fdd3054481/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. 2023-07-24 12:15:21,971 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/0724033c29594920bacf172a08ff3486.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/0724033c29594920bacf172a08ff3486-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/4a1b0a2124fe4972a769ebadde402b77.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/4a1b0a2124fe4972a769ebadde402b77-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/facefb2574884ec39dca1d0bbc3febee.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/facefb2574884ec39dca1d0bbc3febee-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-aede97c2b22f49a4b7797be56223204a, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-8b6c10a14d1d42b2a42745390bcbe281, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-dc24e7440e4849ffb253557dbbfc9577] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/.tmp, totalSize=34.8 K 2023-07-24 12:15:21,971 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/193481541c9d48bfb07a11be0fa1f43b.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/193481541c9d48bfb07a11be0fa1f43b-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/59e63b36c53b463c8e3f6fa553db2f4b.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/59e63b36c53b463c8e3f6fa553db2f4b-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/b2370636c6504c23a0cf52ae01ffb6d4.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/b2370636c6504c23a0cf52ae01ffb6d4-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/2ba017bbab1e409087169dd4001ce0ce.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/2ba017bbab1e409087169dd4001ce0ce-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-334c70b508ad47909996d05ffb25115e, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-0556280e1dd14b4ba397a8cc48d0663e, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-1af444bcfe724d6485ba240a2c869098] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/.tmp, totalSize=34.8 K 2023-07-24 12:15:21,972 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting 0724033c29594920bacf172a08ff3486.ed62c139047156182f9c4cdbe34edfa7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690200919401 2023-07-24 12:15:21,972 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 193481541c9d48bfb07a11be0fa1f43b.ed62c139047156182f9c4cdbe34edfa7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690200919401 2023-07-24 12:15:21,972 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. 2023-07-24 12:15:21,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. 2023-07-24 12:15:21,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. 2023-07-24 12:15:21,973 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 59e63b36c53b463c8e3f6fa553db2f4b.ed62c139047156182f9c4cdbe34edfa7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690200919588 2023-07-24 12:15:21,973 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d6d1a02c2c1c2259a534e0cfc7642c20, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.', STARTKEY => '', ENDKEY => '60'} 2023-07-24 12:15:21,973 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting 4a1b0a2124fe4972a769ebadde402b77.ed62c139047156182f9c4cdbe34edfa7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690200919588 2023-07-24 12:15:21,973 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=76 updating hbase:meta row=7d24409abbbbe6199617e8fdd3054481, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:21,973 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690200921973"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200921973"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200921973"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200921973"}]},"ts":"1690200921973"} 2023-07-24 12:15:21,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:21,974 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting b2370636c6504c23a0cf52ae01ffb6d4.ed62c139047156182f9c4cdbe34edfa7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690200919770 2023-07-24 12:15:21,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionDeletedRowsDaughter d6d1a02c2c1c2259a534e0cfc7642c20 2023-07-24 12:15:21,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:21,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for d6d1a02c2c1c2259a534e0cfc7642c20 2023-07-24 12:15:21,974 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for d6d1a02c2c1c2259a534e0cfc7642c20 2023-07-24 12:15:21,974 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 2ba017bbab1e409087169dd4001ce0ce.ed62c139047156182f9c4cdbe34edfa7, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1690200920514 2023-07-24 12:15:21,975 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-334c70b508ad47909996d05ffb25115e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-07-24 12:15:21,975 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-0556280e1dd14b4ba397a8cc48d0663e, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-07-24 12:15:21,976 INFO [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d6d1a02c2c1c2259a534e0cfc7642c20 2023-07-24 12:15:21,976 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-1af444bcfe724d6485ba240a2c869098, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-07-24 12:15:21,976 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting facefb2574884ec39dca1d0bbc3febee.ed62c139047156182f9c4cdbe34edfa7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690200919770 2023-07-24 12:15:21,977 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting 5b2f3ae3cde941b5a357a8f4f857d1e7.ed62c139047156182f9c4cdbe34edfa7, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=84, earliestPutTs=1690200920514 2023-07-24 12:15:21,977 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1 2023-07-24 12:15:21,977 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1 2023-07-24 12:15:21,978 INFO [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d6d1a02c2c1c2259a534e0cfc7642c20 columnFamilyName cf1 2023-07-24 12:15:21,978 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-aede97c2b22f49a4b7797be56223204a, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=91, earliestPutTs=9223372036854775807 2023-07-24 12:15:21,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=77, resume processing ppid=76 2023-07-24 12:15:21,978 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=77, ppid=76, state=SUCCESS; OpenRegionProcedure 7d24409abbbbe6199617e8fdd3054481, server=jenkins-hbase6.apache.org,34575,1690200894532 in 240 msec 2023-07-24 12:15:21,978 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-8b6c10a14d1d42b2a42745390bcbe281, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=99, earliestPutTs=9223372036854775807 2023-07-24 12:15:21,979 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-dc24e7440e4849ffb253557dbbfc9577, keycount=1, bloomtype=ROW, size=4.9 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-07-24 12:15:21,980 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=76, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7d24409abbbbe6199617e8fdd3054481, ASSIGN in 398 msec 2023-07-24 12:15:21,991 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/0724033c29594920bacf172a08ff3486.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/0724033c29594920bacf172a08ff3486-bottom 2023-07-24 12:15:21,998 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/4a1b0a2124fe4972a769ebadde402b77.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/4a1b0a2124fe4972a769ebadde402b77-bottom 2023-07-24 12:15:22,001 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] throttle.PressureAwareThroughputController(145): 7d24409abbbbe6199617e8fdd3054481#cf2#compaction#72 average throughput is 0.00 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:22,002 INFO [StoreFileOpener-cf1-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 5b2f3ae3cde941b5a357a8f4f857d1e7.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,002 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7-bottom 2023-07-24 12:15:22,004 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7d24409abbbbe6199617e8fdd3054481#cf1#compaction#73 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:22,009 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/facefb2574884ec39dca1d0bbc3febee.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/facefb2574884ec39dca1d0bbc3febee-bottom 2023-07-24 12:15:22,012 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-3c849680a43947559da3bae87e1ed514 2023-07-24 12:15:22,021 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-934c8cda31d2429abcd32850302edc0b 2023-07-24 12:15:22,023 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 12:15:22,034 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-f068438ae0bf4f42b1e48c57a80e0a04 2023-07-24 12:15:22,034 INFO [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(310): Store=d6d1a02c2c1c2259a534e0cfc7642c20/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:22,034 INFO [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d6d1a02c2c1c2259a534e0cfc7642c20 2023-07-24 12:15:22,036 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2 2023-07-24 12:15:22,036 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2 2023-07-24 12:15:22,036 INFO [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d6d1a02c2c1c2259a534e0cfc7642c20 columnFamilyName cf2 2023-07-24 12:15:22,037 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for ec0b0134869f4d068ae10eb7ceb5ca47: NONE, but ROW specified in column family configuration 2023-07-24 12:15:22,039 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/.tmp/cf2/ec0b0134869f4d068ae10eb7ceb5ca47 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/ec0b0134869f4d068ae10eb7ceb5ca47 2023-07-24 12:15:22,053 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 7c292aa0cc414ebabba4cc682975d9f8: NONE, but ROW specified in column family configuration 2023-07-24 12:15:22,054 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/.tmp/cf1/7c292aa0cc414ebabba4cc682975d9f8 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/7c292aa0cc414ebabba4cc682975d9f8 2023-07-24 12:15:22,058 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for ec0b0134869f4d068ae10eb7ceb5ca47: NONE, but ROW specified in column family configuration 2023-07-24 12:15:22,059 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 7c292aa0cc414ebabba4cc682975d9f8: NONE, but ROW specified in column family configuration 2023-07-24 12:15:22,060 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7d24409abbbbe6199617e8fdd3054481/cf2 of 7d24409abbbbe6199617e8fdd3054481 into ec0b0134869f4d068ae10eb7ceb5ca47(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:22,060 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7d24409abbbbe6199617e8fdd3054481: 2023-07-24 12:15:22,060 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481., storeName=7d24409abbbbe6199617e8fdd3054481/cf2, priority=9, startTime=1690200921969; duration=0sec 2023-07-24 12:15:22,060 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:22,060 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 7d24409abbbbe6199617e8fdd3054481/cf1 of 7d24409abbbbe6199617e8fdd3054481 into 7c292aa0cc414ebabba4cc682975d9f8(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:22,060 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7d24409abbbbe6199617e8fdd3054481: 2023-07-24 12:15:22,060 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481., storeName=7d24409abbbbe6199617e8fdd3054481/cf1, priority=9, startTime=1690200921969; duration=0sec 2023-07-24 12:15:22,060 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:22,062 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/193481541c9d48bfb07a11be0fa1f43b.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/193481541c9d48bfb07a11be0fa1f43b-bottom 2023-07-24 12:15:22,069 INFO [StoreFileOpener-cf2-1] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2ba017bbab1e409087169dd4001ce0ce.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,069 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/2ba017bbab1e409087169dd4001ce0ce.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/2ba017bbab1e409087169dd4001ce0ce-bottom 2023-07-24 12:15:22,073 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/59e63b36c53b463c8e3f6fa553db2f4b.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/59e63b36c53b463c8e3f6fa553db2f4b-bottom 2023-07-24 12:15:22,076 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/b2370636c6504c23a0cf52ae01ffb6d4.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/b2370636c6504c23a0cf52ae01ffb6d4-bottom 2023-07-24 12:15:22,080 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-0db122c7c0f84a56b5c228994731bbb0 2023-07-24 12:15:22,083 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-1dff5c81a99942fa878626401a45997e 2023-07-24 12:15:22,091 DEBUG [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-ec16a8641a10453596dedcf56d2c6d25 2023-07-24 12:15:22,091 INFO [StoreOpener-d6d1a02c2c1c2259a534e0cfc7642c20-1] regionserver.HStore(310): Store=d6d1a02c2c1c2259a534e0cfc7642c20/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:22,092 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20 2023-07-24 12:15:22,093 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20 2023-07-24 12:15:22,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionDeletedRowsDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:15:22,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for d6d1a02c2c1c2259a534e0cfc7642c20 2023-07-24 12:15:22,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened d6d1a02c2c1c2259a534e0cfc7642c20; next sequenceid=107; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=67581695, jitterRate=0.007045730948448181}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:15:22,096 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for d6d1a02c2c1c2259a534e0cfc7642c20: 2023-07-24 12:15:22,097 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20., pid=78, masterSystemTime=1690200921886 2023-07-24 12:15:22,097 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:22,099 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:15:22,099 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:15:22,099 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. 2023-07-24 12:15:22,099 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:22,099 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1912): d6d1a02c2c1c2259a534e0cfc7642c20/cf1 is initiating minor compaction (all files) 2023-07-24 12:15:22,099 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. 2023-07-24 12:15:22,099 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d6d1a02c2c1c2259a534e0cfc7642c20/cf1 in testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. 2023-07-24 12:15:22,099 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1912): d6d1a02c2c1c2259a534e0cfc7642c20/cf2 is initiating minor compaction (all files) 2023-07-24 12:15:22,099 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegion(2259): Starting compaction of d6d1a02c2c1c2259a534e0cfc7642c20/cf2 in testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. 2023-07-24 12:15:22,099 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/0724033c29594920bacf172a08ff3486.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/0724033c29594920bacf172a08ff3486-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/4a1b0a2124fe4972a769ebadde402b77.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/4a1b0a2124fe4972a769ebadde402b77-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/facefb2574884ec39dca1d0bbc3febee.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/facefb2574884ec39dca1d0bbc3febee-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-934c8cda31d2429abcd32850302edc0b, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-f068438ae0bf4f42b1e48c57a80e0a04, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-3c849680a43947559da3bae87e1ed514, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7-bottom] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/.tmp, totalSize=35.0 K 2023-07-24 12:15:22,100 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/193481541c9d48bfb07a11be0fa1f43b.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/193481541c9d48bfb07a11be0fa1f43b-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/59e63b36c53b463c8e3f6fa553db2f4b.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/59e63b36c53b463c8e3f6fa553db2f4b-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/b2370636c6504c23a0cf52ae01ffb6d4.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/b2370636c6504c23a0cf52ae01ffb6d4-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-1dff5c81a99942fa878626401a45997e, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-0db122c7c0f84a56b5c228994731bbb0, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-ec16a8641a10453596dedcf56d2c6d25, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/2ba017bbab1e409087169dd4001ce0ce.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/2ba017bbab1e409087169dd4001ce0ce-bottom] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/.tmp, totalSize=35.0 K 2023-07-24 12:15:22,100 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting 0724033c29594920bacf172a08ff3486.ed62c139047156182f9c4cdbe34edfa7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690200919401 2023-07-24 12:15:22,101 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 193481541c9d48bfb07a11be0fa1f43b.ed62c139047156182f9c4cdbe34edfa7, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690200919401 2023-07-24 12:15:22,101 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. 2023-07-24 12:15:22,101 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. 2023-07-24 12:15:22,102 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting 4a1b0a2124fe4972a769ebadde402b77.ed62c139047156182f9c4cdbe34edfa7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690200919588 2023-07-24 12:15:22,102 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 59e63b36c53b463c8e3f6fa553db2f4b.ed62c139047156182f9c4cdbe34edfa7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690200919588 2023-07-24 12:15:22,102 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=75 updating hbase:meta row=d6d1a02c2c1c2259a534e0cfc7642c20, regionState=OPEN, openSeqNum=107, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:22,102 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690200922102"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200922102"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200922102"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200922102"}]},"ts":"1690200922102"} 2023-07-24 12:15:22,102 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting facefb2574884ec39dca1d0bbc3febee.ed62c139047156182f9c4cdbe34edfa7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690200919770 2023-07-24 12:15:22,103 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting b2370636c6504c23a0cf52ae01ffb6d4.ed62c139047156182f9c4cdbe34edfa7, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690200919770 2023-07-24 12:15:22,103 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-934c8cda31d2429abcd32850302edc0b, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690200919966 2023-07-24 12:15:22,104 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-1dff5c81a99942fa878626401a45997e, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690200919966 2023-07-24 12:15:22,106 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-f068438ae0bf4f42b1e48c57a80e0a04, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690200920155 2023-07-24 12:15:22,106 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-0db122c7c0f84a56b5c228994731bbb0, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690200920155 2023-07-24 12:15:22,108 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-3c849680a43947559da3bae87e1ed514, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690200920335 2023-07-24 12:15:22,108 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-ec16a8641a10453596dedcf56d2c6d25, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690200920335 2023-07-24 12:15:22,108 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting 5b2f3ae3cde941b5a357a8f4f857d1e7.ed62c139047156182f9c4cdbe34edfa7, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690200920514 2023-07-24 12:15:22,109 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 2ba017bbab1e409087169dd4001ce0ce.ed62c139047156182f9c4cdbe34edfa7, keycount=4, bloomtype=ROW, size=5.1 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690200920514 2023-07-24 12:15:22,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=78, resume processing ppid=75 2023-07-24 12:15:22,112 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=78, ppid=75, state=SUCCESS; OpenRegionProcedure d6d1a02c2c1c2259a534e0cfc7642c20, server=jenkins-hbase6.apache.org,34575,1690200894532 in 366 msec 2023-07-24 12:15:22,114 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=75, resume processing ppid=72 2023-07-24 12:15:22,114 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=75, ppid=72, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d6d1a02c2c1c2259a534e0cfc7642c20, ASSIGN in 532 msec 2023-07-24 12:15:22,116 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 in 1.0560 sec 2023-07-24 12:15:22,128 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] throttle.PressureAwareThroughputController(145): d6d1a02c2c1c2259a534e0cfc7642c20#cf2#compaction#75 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:22,128 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] throttle.PressureAwareThroughputController(145): d6d1a02c2c1c2259a534e0cfc7642c20#cf1#compaction#74 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:22,155 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/.tmp/cf2/c466a68975354a209d85ae2179f95787 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/c466a68975354a209d85ae2179f95787 2023-07-24 12:15:22,156 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/.tmp/cf1/25feb46745c44a48ba742d6102b381d0 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/25feb46745c44a48ba742d6102b381d0 2023-07-24 12:15:22,161 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d6d1a02c2c1c2259a534e0cfc7642c20/cf2 of d6d1a02c2c1c2259a534e0cfc7642c20 into c466a68975354a209d85ae2179f95787(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:22,161 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for d6d1a02c2c1c2259a534e0cfc7642c20: 2023-07-24 12:15:22,161 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20., storeName=d6d1a02c2c1c2259a534e0cfc7642c20/cf2, priority=9, startTime=1690200922099; duration=0sec 2023-07-24 12:15:22,162 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:22,163 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in d6d1a02c2c1c2259a534e0cfc7642c20/cf1 of d6d1a02c2c1c2259a534e0cfc7642c20 into 25feb46745c44a48ba742d6102b381d0(size=6.8 K), total size for store is 6.8 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:22,163 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d6d1a02c2c1c2259a534e0cfc7642c20: 2023-07-24 12:15:22,163 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20., storeName=d6d1a02c2c1c2259a534e0cfc7642c20/cf1, priority=9, startTime=1690200922097; duration=0sec 2023-07-24 12:15:22,163 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:22,321 DEBUG [Listener at localhost/37279] ipc.RpcConnection(124): Using SIMPLE authentication for service=AdminService, sasl=false 2023-07-24 12:15:22,325 INFO [RS-EventLoopGroup-4-2] ipc.ServerRpcConnection(540): Connection from 172.31.2.20:46698, version=2.4.18-SNAPSHOT, sasl=false, ugi=jenkins (auth:SIMPLE), service=AdminService 2023-07-24 12:15:22,325 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. 2023-07-24 12:15:22,326 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-07-24 12:15:22,326 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.HStore(1912): d6d1a02c2c1c2259a534e0cfc7642c20/cf1 is initiating major compaction (all files) 2023-07-24 12:15:22,326 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@5178e8bf; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:22,326 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of d6d1a02c2c1c2259a534e0cfc7642c20/cf1 in testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. 2023-07-24 12:15:22,326 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-07-24 12:15:22,327 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/25feb46745c44a48ba742d6102b381d0] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/.tmp, totalSize=6.8 K 2023-07-24 12:15:22,327 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.HStore(1912): d6d1a02c2c1c2259a534e0cfc7642c20/cf2 is initiating major compaction (all files) 2023-07-24 12:15:22,327 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@16813c4f; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 12:15:22,327 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegion(2259): Starting compaction of d6d1a02c2c1c2259a534e0cfc7642c20/cf2 in testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. 2023-07-24 12:15:22,327 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/c466a68975354a209d85ae2179f95787] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/.tmp, totalSize=6.8 K 2023-07-24 12:15:22,327 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting 25feb46745c44a48ba742d6102b381d0, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690200919401 2023-07-24 12:15:22,327 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting c466a68975354a209d85ae2179f95787, keycount=49, bloomtype=ROW, size=6.8 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690200919401 2023-07-24 12:15:22,329 INFO [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34575] regionserver.RSRpcServices(1662): Compacting testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. 2023-07-24 12:15:22,329 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34575] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-07-24 12:15:22,329 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34575] regionserver.HStore(1912): 7d24409abbbbe6199617e8fdd3054481/cf1 is initiating major compaction (all files) 2023-07-24 12:15:22,329 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34575] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@7ce3f460; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 12:15:22,329 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34575] compactions.SortedCompactionPolicy(75): Selecting compaction from 1 store files, 0 compacting, 1 eligible, 16 blocking 2023-07-24 12:15:22,329 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34575] regionserver.HStore(1912): 7d24409abbbbe6199617e8fdd3054481/cf2 is initiating major compaction (all files) 2023-07-24 12:15:22,329 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34575] regionserver.CompactSplit(385): Small Compaction requested: org.apache.hadoop.hbase.regionserver.DefaultStoreEngine$DefaultCompactionContext@13fa28d5; Because: User-triggered major compaction for region testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.; compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-07-24 12:15:22,330 INFO [Listener at localhost/37279] hbase.Waiter(180): Waiting up to [6,000] milli-secs(wait.for.ratio=[1]) 2023-07-24 12:15:22,332 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] throttle.PressureAwareThroughputController(145): d6d1a02c2c1c2259a534e0cfc7642c20#cf2#compaction#77 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:22,332 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] throttle.PressureAwareThroughputController(145): d6d1a02c2c1c2259a534e0cfc7642c20#cf1#compaction#76 average throughput is 1.40 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:22,342 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. is not mergeable because it has references 2023-07-24 12:15:22,342 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34575] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. is not mergeable because it has references 2023-07-24 12:15:22,351 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/.tmp/cf2/ff8ab4a7e956436da8a49b3f7a958473 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/ff8ab4a7e956436da8a49b3f7a958473 2023-07-24 12:15:22,352 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/.tmp/cf1/052451847cfb4dfd8c941c6e6c37da3c as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/052451847cfb4dfd8c941c6e6c37da3c 2023-07-24 12:15:22,356 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in d6d1a02c2c1c2259a534e0cfc7642c20/cf2 of d6d1a02c2c1c2259a534e0cfc7642c20 into ff8ab4a7e956436da8a49b3f7a958473(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:22,356 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in d6d1a02c2c1c2259a534e0cfc7642c20/cf1 of d6d1a02c2c1c2259a534e0cfc7642c20 into 052451847cfb4dfd8c941c6e6c37da3c(size=6.9 K), total size for store is 6.9 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:22,356 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for d6d1a02c2c1c2259a534e0cfc7642c20: 2023-07-24 12:15:22,356 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for d6d1a02c2c1c2259a534e0cfc7642c20: 2023-07-24 12:15:22,358 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20., storeName=cf2, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1690200922327; duration=0sec 2023-07-24 12:15:22,358 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20., storeName=cf1, fileCount=1, fileSize=6.8 K (6.8 K), priority=1, time=1690200922326; duration=0sec 2023-07-24 12:15:22,358 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-07-24 12:15:22,358 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=2), splitQueue=0 2023-07-24 12:15:22,358 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 7d24409abbbbe6199617e8fdd3054481/cf1 in testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. 2023-07-24 12:15:22,358 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 7d24409abbbbe6199617e8fdd3054481/cf2 in testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. 2023-07-24 12:15:22,359 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/7c292aa0cc414ebabba4cc682975d9f8] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/.tmp, totalSize=5.0 K 2023-07-24 12:15:22,359 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/ec0b0134869f4d068ae10eb7ceb5ca47] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/.tmp, totalSize=5.0 K 2023-07-24 12:15:22,359 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 7c292aa0cc414ebabba4cc682975d9f8, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-07-24 12:15:22,359 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting ec0b0134869f4d068ae10eb7ceb5ca47, keycount=0, bloomtype=NONE, size=5.0 K, encoding=NONE, compression=NONE, seqNum=103, earliestPutTs=9223372036854775807 2023-07-24 12:15:22,362 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] throttle.PressureAwareThroughputController(145): 7d24409abbbbe6199617e8fdd3054481#cf1#compaction#78 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:22,362 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] throttle.PressureAwareThroughputController(145): 7d24409abbbbe6199617e8fdd3054481#cf2#compaction#79 average throughput is NaN MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:15:22,391 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 1ba296aa26d6422fbfa758ea1334b04c: NONE, but ROW specified in column family configuration 2023-07-24 12:15:22,391 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for e2462b36131e440fa8832da68dde4dd0: NONE, but ROW specified in column family configuration 2023-07-24 12:15:22,392 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/.tmp/cf1/1ba296aa26d6422fbfa758ea1334b04c as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/1ba296aa26d6422fbfa758ea1334b04c 2023-07-24 12:15:22,392 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/.tmp/cf2/e2462b36131e440fa8832da68dde4dd0 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/e2462b36131e440fa8832da68dde4dd0 2023-07-24 12:15:22,396 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for 1ba296aa26d6422fbfa758ea1334b04c: NONE, but ROW specified in column family configuration 2023-07-24 12:15:22,396 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStoreFile(437): HFile Bloom filter type for e2462b36131e440fa8832da68dde4dd0: NONE, but ROW specified in column family configuration 2023-07-24 12:15:22,397 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 7d24409abbbbe6199617e8fdd3054481/cf2 of 7d24409abbbbe6199617e8fdd3054481 into e2462b36131e440fa8832da68dde4dd0(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:22,397 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1652): Completed major compaction of 1 (all) file(s) in 7d24409abbbbe6199617e8fdd3054481/cf1 of 7d24409abbbbe6199617e8fdd3054481 into 1ba296aa26d6422fbfa758ea1334b04c(size=5.0 K), total size for store is 5.0 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:15:22,397 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7d24409abbbbe6199617e8fdd3054481: 2023-07-24 12:15:22,397 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 7d24409abbbbe6199617e8fdd3054481: 2023-07-24 12:15:22,397 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481., storeName=cf2, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1690200922329; duration=0sec 2023-07-24 12:15:22,397 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction Request=regionName=testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481., storeName=cf1, fileCount=1, fileSize=5.0 K (5.0 K), priority=1, time=1690200922329; duration=0sec 2023-07-24 12:15:22,397 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:22,397 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:15:22,450 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. is not mergeable because it has references 2023-07-24 12:15:22,451 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=2,queue=1,port=34575] regionserver.HRegion(1451): Region testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. is not mergeable because it has references 2023-07-24 12:15:22,458 DEBUG [Listener at localhost/37279] client.ClientScanner(240): Advancing internal scanner to startKey at '60', inclusive 2023-07-24 12:15:22,459 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:15:22,460 INFO [Listener at localhost/37279] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:22,461 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$11(2418): Client=jenkins//172.31.2.20 disable testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:22,462 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=79, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:22,464 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-24 12:15:22,464 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200922464"}]},"ts":"1690200922464"} 2023-07-24 12:15:22,465 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLING in hbase:meta 2023-07-24 12:15:22,467 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLING 2023-07-24 12:15:22,467 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d6d1a02c2c1c2259a534e0cfc7642c20, UNASSIGN}, {pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7d24409abbbbe6199617e8fdd3054481, UNASSIGN}] 2023-07-24 12:15:22,468 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=81, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7d24409abbbbe6199617e8fdd3054481, UNASSIGN 2023-07-24 12:15:22,469 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=7d24409abbbbe6199617e8fdd3054481, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:22,469 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690200922469"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200922469"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200922469"}]},"ts":"1690200922469"} 2023-07-24 12:15:22,470 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=82, ppid=81, state=RUNNABLE; CloseRegionProcedure 7d24409abbbbe6199617e8fdd3054481, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:15:22,472 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=80, ppid=79, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d6d1a02c2c1c2259a534e0cfc7642c20, UNASSIGN 2023-07-24 12:15:22,473 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=d6d1a02c2c1c2259a534e0cfc7642c20, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:22,473 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690200922473"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200922473"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200922473"}]},"ts":"1690200922473"} 2023-07-24 12:15:22,474 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=83, ppid=80, state=RUNNABLE; CloseRegionProcedure d6d1a02c2c1c2259a534e0cfc7642c20, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:15:22,565 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-24 12:15:22,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close d6d1a02c2c1c2259a534e0cfc7642c20 2023-07-24 12:15:22,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing d6d1a02c2c1c2259a534e0cfc7642c20, disabling compactions & flushes 2023-07-24 12:15:22,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. 2023-07-24 12:15:22,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. 2023-07-24 12:15:22,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. after waiting 0 ms 2023-07-24 12:15:22,622 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. 2023-07-24 12:15:22,623 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/0724033c29594920bacf172a08ff3486.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/0724033c29594920bacf172a08ff3486-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/4a1b0a2124fe4972a769ebadde402b77.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/4a1b0a2124fe4972a769ebadde402b77-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/facefb2574884ec39dca1d0bbc3febee.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/facefb2574884ec39dca1d0bbc3febee-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-934c8cda31d2429abcd32850302edc0b, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-f068438ae0bf4f42b1e48c57a80e0a04, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-3c849680a43947559da3bae87e1ed514, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/25feb46745c44a48ba742d6102b381d0, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7-bottom] to archive 2023-07-24 12:15:22,624 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:15:22,625 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/0724033c29594920bacf172a08ff3486.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/0724033c29594920bacf172a08ff3486.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,626 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/4a1b0a2124fe4972a769ebadde402b77.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/4a1b0a2124fe4972a769ebadde402b77.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,628 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/facefb2574884ec39dca1d0bbc3febee.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/facefb2574884ec39dca1d0bbc3febee.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,629 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-934c8cda31d2429abcd32850302edc0b to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-934c8cda31d2429abcd32850302edc0b 2023-07-24 12:15:22,630 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-f068438ae0bf4f42b1e48c57a80e0a04 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-f068438ae0bf4f42b1e48c57a80e0a04 2023-07-24 12:15:22,631 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-3c849680a43947559da3bae87e1ed514 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-3c849680a43947559da3bae87e1ed514 2023-07-24 12:15:22,632 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/25feb46745c44a48ba742d6102b381d0 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/25feb46745c44a48ba742d6102b381d0 2023-07-24 12:15:22,633 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,635 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/193481541c9d48bfb07a11be0fa1f43b.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/193481541c9d48bfb07a11be0fa1f43b-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/59e63b36c53b463c8e3f6fa553db2f4b.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/59e63b36c53b463c8e3f6fa553db2f4b-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/b2370636c6504c23a0cf52ae01ffb6d4.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/b2370636c6504c23a0cf52ae01ffb6d4-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-1dff5c81a99942fa878626401a45997e, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-0db122c7c0f84a56b5c228994731bbb0, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-ec16a8641a10453596dedcf56d2c6d25, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/c466a68975354a209d85ae2179f95787, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/2ba017bbab1e409087169dd4001ce0ce.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/2ba017bbab1e409087169dd4001ce0ce-bottom] to archive 2023-07-24 12:15:22,635 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:15:22,637 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/193481541c9d48bfb07a11be0fa1f43b.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/193481541c9d48bfb07a11be0fa1f43b.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,638 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/59e63b36c53b463c8e3f6fa553db2f4b.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/59e63b36c53b463c8e3f6fa553db2f4b.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,639 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/b2370636c6504c23a0cf52ae01ffb6d4.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/b2370636c6504c23a0cf52ae01ffb6d4.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,640 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-1dff5c81a99942fa878626401a45997e to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-1dff5c81a99942fa878626401a45997e 2023-07-24 12:15:22,641 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-0db122c7c0f84a56b5c228994731bbb0 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-0db122c7c0f84a56b5c228994731bbb0 2023-07-24 12:15:22,642 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-ec16a8641a10453596dedcf56d2c6d25 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-ec16a8641a10453596dedcf56d2c6d25 2023-07-24 12:15:22,643 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/c466a68975354a209d85ae2179f95787 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/c466a68975354a209d85ae2179f95787 2023-07-24 12:15:22,644 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/2ba017bbab1e409087169dd4001ce0ce.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/2ba017bbab1e409087169dd4001ce0ce.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-07-24 12:15:22,647 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:22,649 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20. 2023-07-24 12:15:22,649 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for d6d1a02c2c1c2259a534e0cfc7642c20: 2023-07-24 12:15:22,650 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed d6d1a02c2c1c2259a534e0cfc7642c20 2023-07-24 12:15:22,650 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:22,650 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 7d24409abbbbe6199617e8fdd3054481, disabling compactions & flushes 2023-07-24 12:15:22,650 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. 2023-07-24 12:15:22,650 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. 2023-07-24 12:15:22,650 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. after waiting 0 ms 2023-07-24 12:15:22,650 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. 2023-07-24 12:15:22,650 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=80 updating hbase:meta row=d6d1a02c2c1c2259a534e0cfc7642c20, regionState=CLOSED 2023-07-24 12:15:22,651 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690200922650"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200922650"}]},"ts":"1690200922650"} 2023-07-24 12:15:22,651 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/0724033c29594920bacf172a08ff3486.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/0724033c29594920bacf172a08ff3486-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/4a1b0a2124fe4972a769ebadde402b77.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/4a1b0a2124fe4972a769ebadde402b77-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/facefb2574884ec39dca1d0bbc3febee.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/facefb2574884ec39dca1d0bbc3febee-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-aede97c2b22f49a4b7797be56223204a, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-8b6c10a14d1d42b2a42745390bcbe281, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/7c292aa0cc414ebabba4cc682975d9f8, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-dc24e7440e4849ffb253557dbbfc9577] to archive 2023-07-24 12:15:22,652 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:15:22,654 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=83, resume processing ppid=80 2023-07-24 12:15:22,654 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=83, ppid=80, state=SUCCESS; CloseRegionProcedure d6d1a02c2c1c2259a534e0cfc7642c20, server=jenkins-hbase6.apache.org,34575,1690200894532 in 178 msec 2023-07-24 12:15:22,654 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/0724033c29594920bacf172a08ff3486.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/0724033c29594920bacf172a08ff3486.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,655 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=80, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=d6d1a02c2c1c2259a534e0cfc7642c20, UNASSIGN in 187 msec 2023-07-24 12:15:22,655 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/4a1b0a2124fe4972a769ebadde402b77.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/4a1b0a2124fe4972a769ebadde402b77.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,656 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/facefb2574884ec39dca1d0bbc3febee.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/facefb2574884ec39dca1d0bbc3febee.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,657 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,658 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-aede97c2b22f49a4b7797be56223204a to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-aede97c2b22f49a4b7797be56223204a 2023-07-24 12:15:22,659 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-8b6c10a14d1d42b2a42745390bcbe281 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-8b6c10a14d1d42b2a42745390bcbe281 2023-07-24 12:15:22,660 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/7c292aa0cc414ebabba4cc682975d9f8 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/7c292aa0cc414ebabba4cc682975d9f8 2023-07-24 12:15:22,662 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-dc24e7440e4849ffb253557dbbfc9577 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-dc24e7440e4849ffb253557dbbfc9577 2023-07-24 12:15:22,663 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/193481541c9d48bfb07a11be0fa1f43b.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/193481541c9d48bfb07a11be0fa1f43b-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/59e63b36c53b463c8e3f6fa553db2f4b.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/59e63b36c53b463c8e3f6fa553db2f4b-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/b2370636c6504c23a0cf52ae01ffb6d4.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/b2370636c6504c23a0cf52ae01ffb6d4-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/2ba017bbab1e409087169dd4001ce0ce.ed62c139047156182f9c4cdbe34edfa7->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/2ba017bbab1e409087169dd4001ce0ce-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-334c70b508ad47909996d05ffb25115e, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-0556280e1dd14b4ba397a8cc48d0663e, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/ec0b0134869f4d068ae10eb7ceb5ca47, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-1af444bcfe724d6485ba240a2c869098] to archive 2023-07-24 12:15:22,664 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:15:22,665 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/193481541c9d48bfb07a11be0fa1f43b.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/193481541c9d48bfb07a11be0fa1f43b.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,666 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/59e63b36c53b463c8e3f6fa553db2f4b.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/59e63b36c53b463c8e3f6fa553db2f4b.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,667 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/b2370636c6504c23a0cf52ae01ffb6d4.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/b2370636c6504c23a0cf52ae01ffb6d4.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,668 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/2ba017bbab1e409087169dd4001ce0ce.ed62c139047156182f9c4cdbe34edfa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/2ba017bbab1e409087169dd4001ce0ce.ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,669 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-334c70b508ad47909996d05ffb25115e to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-334c70b508ad47909996d05ffb25115e 2023-07-24 12:15:22,670 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-0556280e1dd14b4ba397a8cc48d0663e to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-0556280e1dd14b4ba397a8cc48d0663e 2023-07-24 12:15:22,672 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/ec0b0134869f4d068ae10eb7ceb5ca47 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/ec0b0134869f4d068ae10eb7ceb5ca47 2023-07-24 12:15:22,673 DEBUG [StoreCloser-testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-1af444bcfe724d6485ba240a2c869098 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/testSplitTableRegionDeletedRowsDaughter=ed62c139047156182f9c4cdbe34edfa7-1af444bcfe724d6485ba240a2c869098 2023-07-24 12:15:22,677 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/recovered.edits/113.seqid, newMaxSeqId=113, maxSeqId=106 2023-07-24 12:15:22,677 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:22,678 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481. 2023-07-24 12:15:22,678 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 7d24409abbbbe6199617e8fdd3054481: 2023-07-24 12:15:22,680 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:22,680 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=81 updating hbase:meta row=7d24409abbbbe6199617e8fdd3054481, regionState=CLOSED 2023-07-24 12:15:22,680 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.","families":{"info":[{"qualifier":"regioninfo","vlen":75,"tag":[],"timestamp":"1690200922680"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200922680"}]},"ts":"1690200922680"} 2023-07-24 12:15:22,683 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=82, resume processing ppid=81 2023-07-24 12:15:22,683 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=82, ppid=81, state=SUCCESS; CloseRegionProcedure 7d24409abbbbe6199617e8fdd3054481, server=jenkins-hbase6.apache.org,34575,1690200894532 in 211 msec 2023-07-24 12:15:22,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=81, resume processing ppid=79 2023-07-24 12:15:22,684 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=81, ppid=79, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionDeletedRowsDaughter, region=7d24409abbbbe6199617e8fdd3054481, UNASSIGN in 216 msec 2023-07-24 12:15:22,685 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200922685"}]},"ts":"1690200922685"} 2023-07-24 12:15:22,686 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionDeletedRowsDaughter, state=DISABLED in hbase:meta 2023-07-24 12:15:22,689 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionDeletedRowsDaughter to state=DISABLED 2023-07-24 12:15:22,690 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 228 msec 2023-07-24 12:15:22,766 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=79 2023-07-24 12:15:22,766 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 79 completed 2023-07-24 12:15:22,767 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$5(2228): Client=jenkins//172.31.2.20 delete testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:22,767 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:22,769 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=84, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:22,770 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=84, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:22,770 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-07-24 12:15:22,774 DEBUG [HFileArchiver-8] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,774 DEBUG [HFileArchiver-2] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20 2023-07-24 12:15:22,774 DEBUG [HFileArchiver-5] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:22,776 DEBUG [HFileArchiver-8] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/recovered.edits] 2023-07-24 12:15:22,776 DEBUG [HFileArchiver-2] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/recovered.edits] 2023-07-24 12:15:22,776 DEBUG [HFileArchiver-5] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/recovered.edits] 2023-07-24 12:15:22,779 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/0724033c29594920bacf172a08ff3486 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/0724033c29594920bacf172a08ff3486 2023-07-24 12:15:22,779 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/052451847cfb4dfd8c941c6e6c37da3c to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf1/052451847cfb4dfd8c941c6e6c37da3c 2023-07-24 12:15:22,780 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/1ba296aa26d6422fbfa758ea1334b04c to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf1/1ba296aa26d6422fbfa758ea1334b04c 2023-07-24 12:15:22,782 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/3c849680a43947559da3bae87e1ed514 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/3c849680a43947559da3bae87e1ed514 2023-07-24 12:15:22,783 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/e2462b36131e440fa8832da68dde4dd0 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/cf2/e2462b36131e440fa8832da68dde4dd0 2023-07-24 12:15:22,783 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/ff8ab4a7e956436da8a49b3f7a958473 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/cf2/ff8ab4a7e956436da8a49b3f7a958473 2023-07-24 12:15:22,783 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/4a1b0a2124fe4972a769ebadde402b77 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/4a1b0a2124fe4972a769ebadde402b77 2023-07-24 12:15:22,785 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/5b2f3ae3cde941b5a357a8f4f857d1e7 2023-07-24 12:15:22,786 DEBUG [HFileArchiver-2] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/recovered.edits/113.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20/recovered.edits/113.seqid 2023-07-24 12:15:22,786 DEBUG [HFileArchiver-5] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/recovered.edits/113.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481/recovered.edits/113.seqid 2023-07-24 12:15:22,786 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/8b6c10a14d1d42b2a42745390bcbe281 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/8b6c10a14d1d42b2a42745390bcbe281 2023-07-24 12:15:22,787 DEBUG [HFileArchiver-5] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:15:22,787 DEBUG [HFileArchiver-2] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/d6d1a02c2c1c2259a534e0cfc7642c20 2023-07-24 12:15:22,787 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/934c8cda31d2429abcd32850302edc0b to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/934c8cda31d2429abcd32850302edc0b 2023-07-24 12:15:22,789 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/aede97c2b22f49a4b7797be56223204a to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/aede97c2b22f49a4b7797be56223204a 2023-07-24 12:15:22,790 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/dc24e7440e4849ffb253557dbbfc9577 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/dc24e7440e4849ffb253557dbbfc9577 2023-07-24 12:15:22,791 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/f068438ae0bf4f42b1e48c57a80e0a04 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/f068438ae0bf4f42b1e48c57a80e0a04 2023-07-24 12:15:22,792 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/facefb2574884ec39dca1d0bbc3febee to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf1/facefb2574884ec39dca1d0bbc3febee 2023-07-24 12:15:22,794 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/0556280e1dd14b4ba397a8cc48d0663e to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/0556280e1dd14b4ba397a8cc48d0663e 2023-07-24 12:15:22,796 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/0db122c7c0f84a56b5c228994731bbb0 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/0db122c7c0f84a56b5c228994731bbb0 2023-07-24 12:15:22,797 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/193481541c9d48bfb07a11be0fa1f43b to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/193481541c9d48bfb07a11be0fa1f43b 2023-07-24 12:15:22,798 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/1af444bcfe724d6485ba240a2c869098 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/1af444bcfe724d6485ba240a2c869098 2023-07-24 12:15:22,799 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/1dff5c81a99942fa878626401a45997e to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/1dff5c81a99942fa878626401a45997e 2023-07-24 12:15:22,800 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/2ba017bbab1e409087169dd4001ce0ce to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/2ba017bbab1e409087169dd4001ce0ce 2023-07-24 12:15:22,801 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/334c70b508ad47909996d05ffb25115e to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/334c70b508ad47909996d05ffb25115e 2023-07-24 12:15:22,802 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/59e63b36c53b463c8e3f6fa553db2f4b to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/59e63b36c53b463c8e3f6fa553db2f4b 2023-07-24 12:15:22,803 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/b2370636c6504c23a0cf52ae01ffb6d4 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/b2370636c6504c23a0cf52ae01ffb6d4 2023-07-24 12:15:22,804 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/ec16a8641a10453596dedcf56d2c6d25 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/cf2/ec16a8641a10453596dedcf56d2c6d25 2023-07-24 12:15:22,806 DEBUG [HFileArchiver-8] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/recovered.edits/106.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7/recovered.edits/106.seqid 2023-07-24 12:15:22,807 DEBUG [HFileArchiver-8] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionDeletedRowsDaughter/ed62c139047156182f9c4cdbe34edfa7 2023-07-24 12:15:22,807 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionDeletedRowsDaughter regions 2023-07-24 12:15:22,809 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=84, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:22,811 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionDeletedRowsDaughter from hbase:meta 2023-07-24 12:15:22,813 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionDeletedRowsDaughter' descriptor. 2023-07-24 12:15:22,813 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=84, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:22,814 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionDeletedRowsDaughter' from region states. 2023-07-24 12:15:22,814 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200922814"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:22,814 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200922814"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:22,814 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200922814"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:22,815 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-24 12:15:22,816 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ed62c139047156182f9c4cdbe34edfa7, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1690200918894.ed62c139047156182f9c4cdbe34edfa7.', STARTKEY => '', ENDKEY => ''}, {ENCODED => d6d1a02c2c1c2259a534e0cfc7642c20, NAME => 'testSplitTableRegionDeletedRowsDaughter,,1690200921059.d6d1a02c2c1c2259a534e0cfc7642c20.', STARTKEY => '', ENDKEY => '60'}, {ENCODED => 7d24409abbbbe6199617e8fdd3054481, NAME => 'testSplitTableRegionDeletedRowsDaughter,60,1690200921059.7d24409abbbbe6199617e8fdd3054481.', STARTKEY => '60', ENDKEY => ''}] 2023-07-24 12:15:22,816 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionDeletedRowsDaughter' as deleted. 2023-07-24 12:15:22,816 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionDeletedRowsDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690200922816"}]},"ts":"9223372036854775807"} 2023-07-24 12:15:22,817 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionDeletedRowsDaughter state from META 2023-07-24 12:15:22,819 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=84, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:15:22,820 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter in 52 msec 2023-07-24 12:15:22,871 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=84 2023-07-24 12:15:22,871 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionDeletedRowsDaughter, procId: 84 completed 2023-07-24 12:15:22,887 INFO [Listener at localhost/37279] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionDeletedRowsDaughter Thread=415 (was 405) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:37996 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-metaLookup-shared--pool-10 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-metaLookup-shared--pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:44954 [Waiting for operation #3] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:60550 [Waiting for operation #10] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:44854 [Waiting for operation #4] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-metaLookup-shared--pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-metaLookup-shared--pool-11 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-17 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-13 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:60952 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:37790 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:44710 [Waiting for operation #11] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:51456 [Waiting for operation #39] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:32894 [Waiting for operation #6] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-5 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:44696 [Waiting for operation #13] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RPCClient-NioEventLoopGroup-6-4 sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hbase.thirdparty.io.netty.channel.nio.SelectedSelectionKeySetSelector.select(SelectedSelectionKeySetSelector.java:62) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.select(NioEventLoop.java:883) org.apache.hbase.thirdparty.io.netty.channel.nio.NioEventLoop.run(NioEventLoop.java:526) org.apache.hbase.thirdparty.io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:997) org.apache.hbase.thirdparty.io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) org.apache.hbase.thirdparty.io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-metaLookup-shared--pool-12 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:44618 [Waiting for operation #36] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-16 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:45664 [Waiting for operation #42] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-14 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: hconnection-0x3d545144-shared-pool-15 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078) java.util.concurrent.LinkedBlockingQueue.poll(LinkedBlockingQueue.java:467) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1073) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:33020 [Waiting for operation #5] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:37536 [Waiting for operation #14] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:38544 [Waiting for operation #18] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: DataXceiver for client DFSClient_NONMAPREDUCE_-338465747_17 at /127.0.0.1:44858 [Waiting for operation #2] sun.nio.ch.EPollArrayWrapper.epollWait(Native Method) sun.nio.ch.EPollArrayWrapper.poll(EPollArrayWrapper.java:269) sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:93) sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:86) sun.nio.ch.SelectorImpl.select(SelectorImpl.java:97) org.apache.hadoop.net.SocketIOWithTimeout$SelectorPool.select(SocketIOWithTimeout.java:335) org.apache.hadoop.net.SocketIOWithTimeout.doIO(SocketIOWithTimeout.java:157) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:161) org.apache.hadoop.net.SocketInputStream.read(SocketInputStream.java:131) java.io.BufferedInputStream.fill(BufferedInputStream.java:246) java.io.BufferedInputStream.read(BufferedInputStream.java:265) java.io.DataInputStream.readShort(DataInputStream.java:312) org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.readOp(Receiver.java:67) org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:269) java.lang.Thread.run(Thread.java:750) Potentially hanging thread: RS:1;jenkins-hbase6:34575-shortCompactions-0 sun.misc.Unsafe.park(Native Method) java.util.concurrent.locks.LockSupport.park(LockSupport.java:175) java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(AbstractQueuedSynchronizer.java:2039) java.util.concurrent.PriorityBlockingQueue.take(PriorityBlockingQueue.java:549) java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1074) java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1134) java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) java.lang.Thread.run(Thread.java:750) - Thread LEAK? -, OpenFileDescriptor=877 (was 871) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=330 (was 330), ProcessCount=168 (was 169), AvailableMemoryMB=1556 (was 1572) 2023-07-24 12:15:22,902 INFO [Listener at localhost/37279] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=415, OpenFileDescriptor=877, MaxFileDescriptor=60000, SystemLoadAverage=330, ProcessCount=168, AvailableMemoryMB=1555 2023-07-24 12:15:22,902 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:15:22,904 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 12:15:22,905 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(492): Client=jenkins//172.31.2.20 set balanceSwitch=false 2023-07-24 12:15:22,906 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:15:22,908 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 12:15:22,908 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 12:15:22,910 DEBUG [HFileArchiver-4] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206 2023-07-24 12:15:22,910 DEBUG [HFileArchiver-4] backup.HFileArchiver(153): Directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206 empty. 2023-07-24 12:15:22,911 DEBUG [HFileArchiver-4] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206 2023-07-24 12:15:22,911 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-07-24 12:15:22,921 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-07-24 12:15:22,922 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(7675): creating {ENCODED => 01a858500fdab441704f5de8ae140206, NAME => 'testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp 2023-07-24 12:15:22,934 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:22,935 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1604): Closing 01a858500fdab441704f5de8ae140206, disabling compactions & flushes 2023-07-24 12:15:22,935 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:15:22,935 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:15:22,935 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. after waiting 0 ms 2023-07-24 12:15:22,935 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:15:22,935 INFO [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:15:22,935 DEBUG [RegionOpenAndInit-testRollbackForSplitTableRegionWithReplica-pool-0] regionserver.HRegion(1558): Region close journal for 01a858500fdab441704f5de8ae140206: 2023-07-24 12:15:22,937 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 12:15:22,937 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690200922937"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200922937"}]},"ts":"1690200922937"} 2023-07-24 12:15:22,939 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 12:15:22,940 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 12:15:22,940 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200922940"}]},"ts":"1690200922940"} 2023-07-24 12:15:22,941 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLING in hbase:meta 2023-07-24 12:15:22,944 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase6.apache.org=0} racks are {/default-rack=0} 2023-07-24 12:15:22,945 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 12:15:22,945 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 12:15:22,945 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 12:15:22,945 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 12:15:22,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=01a858500fdab441704f5de8ae140206, ASSIGN}] 2023-07-24 12:15:22,946 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=01a858500fdab441704f5de8ae140206, ASSIGN 2023-07-24 12:15:22,947 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=86, ppid=85, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=01a858500fdab441704f5de8ae140206, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,36983,1690200894572; forceNewPlan=false, retain=false 2023-07-24 12:15:23,097 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 12:15:23,100 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=01a858500fdab441704f5de8ae140206, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:23,100 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690200923100"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200923100"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200923100"}]},"ts":"1690200923100"} 2023-07-24 12:15:23,101 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=87, ppid=86, state=RUNNABLE; OpenRegionProcedure 01a858500fdab441704f5de8ae140206, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:23,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:15:23,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 01a858500fdab441704f5de8ae140206, NAME => 'testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:15:23,256 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:23,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:23,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,256 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,258 INFO [StoreOpener-01a858500fdab441704f5de8ae140206-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,259 DEBUG [StoreOpener-01a858500fdab441704f5de8ae140206-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206/cf1 2023-07-24 12:15:23,259 DEBUG [StoreOpener-01a858500fdab441704f5de8ae140206-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206/cf1 2023-07-24 12:15:23,260 INFO [StoreOpener-01a858500fdab441704f5de8ae140206-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 01a858500fdab441704f5de8ae140206 columnFamilyName cf1 2023-07-24 12:15:23,261 INFO [StoreOpener-01a858500fdab441704f5de8ae140206-1] regionserver.HStore(310): Store=01a858500fdab441704f5de8ae140206/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:23,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 12:15:23,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 01a858500fdab441704f5de8ae140206; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=59876180, jitterRate=-0.10777539014816284}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 12:15:23,267 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 01a858500fdab441704f5de8ae140206: 2023-07-24 12:15:23,268 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206., pid=87, masterSystemTime=1690200923253 2023-07-24 12:15:23,269 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:15:23,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:15:23,270 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=86 updating hbase:meta row=01a858500fdab441704f5de8ae140206, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:23,270 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690200923269"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200923269"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200923269"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200923269"}]},"ts":"1690200923269"} 2023-07-24 12:15:23,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=87, resume processing ppid=86 2023-07-24 12:15:23,273 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=87, ppid=86, state=SUCCESS; OpenRegionProcedure 01a858500fdab441704f5de8ae140206, server=jenkins-hbase6.apache.org,36983,1690200894572 in 170 msec 2023-07-24 12:15:23,274 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=86, resume processing ppid=85 2023-07-24 12:15:23,274 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=86, ppid=85, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=01a858500fdab441704f5de8ae140206, ASSIGN in 328 msec 2023-07-24 12:15:23,275 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 12:15:23,275 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200923275"}]},"ts":"1690200923275"} 2023-07-24 12:15:23,276 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=ENABLED in hbase:meta 2023-07-24 12:15:23,281 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=85, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 12:15:23,283 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica in 376 msec 2023-07-24 12:15:23,416 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$12(2454): Client=jenkins//172.31.2.20 modify table testRollbackForSplitTableRegionWithReplica from 'testRollbackForSplitTableRegionWithReplica', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} to 'testRollbackForSplitTableRegionWithReplica', {TABLE_ATTRIBUTES => {REGION_REPLICATION => '2'}}, {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'} 2023-07-24 12:15:23,427 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=88, state=RUNNABLE:MODIFY_TABLE_PREPARE; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:15:23,442 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-07-24 12:15:23,443 DEBUG [PEWorker-1] util.FSTableDescriptors(519): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000001 2023-07-24 12:15:23,443 INFO [PEWorker-1] util.FSTableDescriptors(293): Updated tableinfo=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/.tabledesc/.tableinfo.0000000002 2023-07-24 12:15:23,449 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=89, ppid=88, state=RUNNABLE:REOPEN_TABLE_REGIONS_GET_REGIONS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica}] 2023-07-24 12:15:23,461 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=01a858500fdab441704f5de8ae140206, REOPEN/MOVE}] 2023-07-24 12:15:23,462 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=01a858500fdab441704f5de8ae140206, REOPEN/MOVE 2023-07-24 12:15:23,462 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=01a858500fdab441704f5de8ae140206, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:23,462 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690200923462"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200923462"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200923462"}]},"ts":"1690200923462"} 2023-07-24 12:15:23,464 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=91, ppid=90, state=RUNNABLE; CloseRegionProcedure 01a858500fdab441704f5de8ae140206, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:23,616 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,617 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 01a858500fdab441704f5de8ae140206, disabling compactions & flushes 2023-07-24 12:15:23,617 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:15:23,617 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:15:23,617 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. after waiting 0 ms 2023-07-24 12:15:23,617 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:15:23,620 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206/recovered.edits/4.seqid, newMaxSeqId=4, maxSeqId=1 2023-07-24 12:15:23,621 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:15:23,621 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:15:23,621 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 01a858500fdab441704f5de8ae140206: 2023-07-24 12:15:23,621 WARN [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(3510): Not adding moved region record: 01a858500fdab441704f5de8ae140206 to self. 2023-07-24 12:15:23,622 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,623 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=01a858500fdab441704f5de8ae140206, regionState=CLOSED 2023-07-24 12:15:23,623 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690200923622"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200923622"}]},"ts":"1690200923622"} 2023-07-24 12:15:23,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=91, resume processing ppid=90 2023-07-24 12:15:23,625 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=91, ppid=90, state=SUCCESS; CloseRegionProcedure 01a858500fdab441704f5de8ae140206, server=jenkins-hbase6.apache.org,36983,1690200894572 in 160 msec 2023-07-24 12:15:23,626 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=90, ppid=89, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=01a858500fdab441704f5de8ae140206, REOPEN/MOVE; state=CLOSED, location=jenkins-hbase6.apache.org,36983,1690200894572; forceNewPlan=false, retain=true 2023-07-24 12:15:23,776 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 12:15:23,776 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=01a858500fdab441704f5de8ae140206, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:23,777 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690200923776"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200923776"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200923776"}]},"ts":"1690200923776"} 2023-07-24 12:15:23,778 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=92, ppid=90, state=RUNNABLE; OpenRegionProcedure 01a858500fdab441704f5de8ae140206, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:15:23,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:15:23,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 01a858500fdab441704f5de8ae140206, NAME => 'testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:15:23,933 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:23,933 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:23,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,934 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,935 INFO [StoreOpener-01a858500fdab441704f5de8ae140206-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,935 DEBUG [StoreOpener-01a858500fdab441704f5de8ae140206-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206/cf1 2023-07-24 12:15:23,936 DEBUG [StoreOpener-01a858500fdab441704f5de8ae140206-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206/cf1 2023-07-24 12:15:23,936 INFO [StoreOpener-01a858500fdab441704f5de8ae140206-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 01a858500fdab441704f5de8ae140206 columnFamilyName cf1 2023-07-24 12:15:23,936 INFO [StoreOpener-01a858500fdab441704f5de8ae140206-1] regionserver.HStore(310): Store=01a858500fdab441704f5de8ae140206/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:15:23,937 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,938 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,939 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 01a858500fdab441704f5de8ae140206 2023-07-24 12:15:23,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 01a858500fdab441704f5de8ae140206; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60254214, jitterRate=-0.10214224457740784}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 12:15:23,940 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 01a858500fdab441704f5de8ae140206: 2023-07-24 12:15:23,941 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206., pid=92, masterSystemTime=1690200923930 2023-07-24 12:15:23,942 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:15:23,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:15:23,943 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=90 updating hbase:meta row=01a858500fdab441704f5de8ae140206, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:15:23,943 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690200923942"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200923942"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200923942"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200923942"}]},"ts":"1690200923942"} 2023-07-24 12:15:23,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=92, resume processing ppid=90 2023-07-24 12:15:23,945 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=92, ppid=90, state=SUCCESS; OpenRegionProcedure 01a858500fdab441704f5de8ae140206, server=jenkins-hbase6.apache.org,36983,1690200894572 in 166 msec 2023-07-24 12:15:23,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=90, resume processing ppid=89 2023-07-24 12:15:23,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=90, ppid=89, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=01a858500fdab441704f5de8ae140206, REOPEN/MOVE in 484 msec 2023-07-24 12:15:23,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=89, resume processing ppid=88 2023-07-24 12:15:23,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=89, ppid=88, state=SUCCESS; ReopenTableRegionsProcedure table=testRollbackForSplitTableRegionWithReplica in 500 msec 2023-07-24 12:15:23,952 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520, ASSIGN}] 2023-07-24 12:15:23,953 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520, ASSIGN 2023-07-24 12:15:23,953 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=93, ppid=88, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520, ASSIGN; state=OFFLINE, location=null; forceNewPlan=false, retain=false 2023-07-24 12:15:24,103 DEBUG [jenkins-hbase6:44983] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase6.apache.org=0} racks are {/default-rack=0} 2023-07-24 12:15:24,104 DEBUG [jenkins-hbase6:44983] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 12:15:24,104 DEBUG [jenkins-hbase6:44983] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 12:15:24,105 DEBUG [jenkins-hbase6:44983] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 12:15:24,105 DEBUG [jenkins-hbase6:44983] balancer.BaseLoadBalancer$Cluster(378): Number of tables=3, number of hosts=1, number of racks=1 2023-07-24 12:15:24,106 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=2e4f3d10799c74f83e99aa427a3ad520, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:15:24,106 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690200924106"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1690200924106"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1690200924106"}]},"ts":"1690200924106"} 2023-07-24 12:15:24,110 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=94, ppid=93, state=RUNNABLE; OpenRegionProcedure 2e4f3d10799c74f83e99aa427a3ad520, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:15:24,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testRollbackForSplitTableRegionWithReplica,,1690200922905_0001.2e4f3d10799c74f83e99aa427a3ad520. 2023-07-24 12:15:24,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 2e4f3d10799c74f83e99aa427a3ad520, NAME => 'testRollbackForSplitTableRegionWithReplica,,1690200922905_0001.2e4f3d10799c74f83e99aa427a3ad520.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1} 2023-07-24 12:15:24,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:15:24,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRollbackForSplitTableRegionWithReplica 2e4f3d10799c74f83e99aa427a3ad520 2023-07-24 12:15:24,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testRollbackForSplitTableRegionWithReplica,,1690200922905_0001.2e4f3d10799c74f83e99aa427a3ad520.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:15:24,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 2e4f3d10799c74f83e99aa427a3ad520 2023-07-24 12:15:24,265 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 2e4f3d10799c74f83e99aa427a3ad520 2023-07-24 12:15:24,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:24,365 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:24,465 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:24,565 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:24,665 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:24,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:24,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:24,966 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:25,066 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:25,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:25,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:25,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:25,467 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:25,567 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:25,667 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:25,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:25,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:25,886 DEBUG [HBase-Metrics2-1] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRollbackForSplitTableRegionWithReplica' 2023-07-24 12:15:25,968 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:26,068 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:26,168 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:26,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:26,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:26,469 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:26,569 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:26,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:26,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:26,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:26,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:27,070 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:27,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:27,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:27,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:27,393 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 12:15:27,471 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:27,571 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:27,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:27,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:27,872 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:27,972 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:28,072 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:28,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:28,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:28,373 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:28,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:28,573 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:28,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:28,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:28,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:28,974 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:29,074 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:29,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:29,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:29,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:29,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:29,575 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:29,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:29,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:29,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:29,976 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:30,076 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:30,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:30,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:30,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:30,477 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:30,577 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:30,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:30,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:30,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:30,978 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:31,078 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:31,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:31,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:31,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:31,479 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:31,579 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:31,679 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:31,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:31,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:31,980 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:32,080 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:32,180 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:32,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:32,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:32,481 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:32,581 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:32,681 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:32,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:32,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:32,982 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:33,082 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:33,182 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:33,282 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:33,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:33,483 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:33,583 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:33,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:33,783 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:33,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:33,984 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:34,084 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:34,184 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:34,284 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:34,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:34,485 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:34,585 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:34,685 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:34,785 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:34,886 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:34,986 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:35,086 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:35,186 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:35,286 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:35,387 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:35,487 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:35,587 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:35,687 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:35,787 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:35,888 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:35,988 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:36,088 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:36,188 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:36,288 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:36,389 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:36,489 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:36,589 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:36,689 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:36,789 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:36,890 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:36,990 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:37,090 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:37,190 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:37,290 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:37,391 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:37,491 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:37,591 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:37,691 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:37,791 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:37,892 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:37,992 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:38,092 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:38,192 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:38,292 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:38,393 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:38,493 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:38,593 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:38,693 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:38,793 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:38,894 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:38,994 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:39,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:39,194 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:39,294 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:39,395 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:39,495 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:39,595 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:39,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:39,795 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:39,896 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:39,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:40,096 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:40,196 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:40,296 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:40,397 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:40,497 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:40,597 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:40,697 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:40,797 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:40,897 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:40,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:41,098 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:41,198 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:41,298 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:41,398 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:41,499 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:41,599 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:41,699 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:41,799 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:41,899 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:42,000 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:42,100 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:42,200 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:42,300 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:42,400 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:42,501 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:42,601 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:42,701 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:42,801 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:42,901 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:43,001 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:43,102 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:43,202 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:43,302 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:43,402 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:43,502 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:43,603 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:43,703 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:43,803 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:43,903 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:44,004 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:44,104 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:44,204 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:44,304 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:44,404 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:44,505 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:44,605 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:44,705 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:44,805 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:44,905 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:45,006 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:45,106 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:45,206 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:45,306 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:45,406 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:45,507 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:45,607 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:45,707 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:45,807 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:45,907 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:46,007 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:46,108 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:46,208 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:46,308 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:46,408 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:46,508 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:46,609 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:46,709 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:46,809 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:46,909 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:47,009 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:47,110 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:47,210 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:47,310 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:47,410 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:47,510 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:47,610 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:47,711 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:47,811 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:47,911 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:48,011 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:48,111 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:48,212 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:48,312 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:48,412 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:48,512 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:48,612 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:48,713 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:48,813 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:48,913 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:49,013 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:49,113 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:49,213 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:49,314 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:49,414 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:49,514 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:49,614 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:49,714 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:49,815 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:49,915 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:50,015 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:50,115 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:50,216 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:50,316 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:50,416 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:50,516 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:50,616 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:50,716 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:50,817 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:50,917 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:51,017 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:51,117 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:51,217 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:51,318 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:51,418 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:51,518 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:51,618 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:51,719 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:51,819 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:51,919 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:52,019 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:52,119 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:52,220 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:52,320 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:52,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:52,520 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:52,620 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:52,721 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:52,821 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:52,921 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:53,021 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:53,122 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:53,222 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:53,322 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:53,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:53,523 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:53,623 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:53,723 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:53,823 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:53,923 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:54,024 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:54,124 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:54,224 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:54,324 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:54,424 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:54,525 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:54,625 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:54,725 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:54,825 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:54,925 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:55,026 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:55,126 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:55,226 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:55,326 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:55,426 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:55,527 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:55,627 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:55,727 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:55,827 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:55,928 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:56,028 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:56,128 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:56,228 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:56,329 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:56,429 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:56,529 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:56,629 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:56,729 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:56,830 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:56,930 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:57,030 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:57,130 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:57,231 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:57,331 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:57,431 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:57,531 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:57,632 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:57,732 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:57,832 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:57,932 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:58,032 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:58,133 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:58,233 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:58,333 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:58,433 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:58,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:58,633 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:58,734 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:58,834 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:58,934 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:59,034 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:59,134 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:59,235 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:59,335 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:59,435 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:59,535 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:59,635 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:59,736 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:59,836 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:15:59,936 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:00,036 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:00,136 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:00,237 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:00,337 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:00,437 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:00,537 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:00,637 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:00,737 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:00,838 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:00,938 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:01,038 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:01,138 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:01,238 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:01,339 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:01,439 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:01,539 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:01,639 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:01,739 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:01,840 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:01,940 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:02,040 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:02,140 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:02,240 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:02,341 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:02,441 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:02,541 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:02,641 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:02,741 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:02,842 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:02,942 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:03,042 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:03,142 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:03,242 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:03,343 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:03,443 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:03,543 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:03,643 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:03,744 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:03,844 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:03,944 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:04,044 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:04,144 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:04,245 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:04,345 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:04,445 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:04,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:04,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:04,745 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:04,846 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:04,946 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:05,046 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:05,146 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:05,246 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:05,347 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:05,447 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:05,547 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:05,647 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:05,747 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:05,848 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:05,948 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:06,048 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:06,148 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:06,248 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:06,349 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:06,449 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:06,549 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:06,649 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:06,750 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:06,850 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:06,950 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:07,050 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:07,150 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:07,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:07,351 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:07,451 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:07,551 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:07,652 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:07,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:07,852 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:07,952 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:08,052 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:08,153 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:08,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:08,353 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:08,453 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:08,553 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:08,654 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:08,754 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:08,854 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:08,954 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:09,054 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:09,155 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:09,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:09,355 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:09,455 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:09,556 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:09,656 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:09,756 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:09,856 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:09,956 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:10,057 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:10,157 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:10,257 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:10,357 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:10,457 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:10,558 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:10,658 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:10,758 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:10,858 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:10,958 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:11,059 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:11,159 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:11,259 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:11,359 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:11,459 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:11,560 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:11,660 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:11,760 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:11,860 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:11,960 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:12,061 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:12,161 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:12,261 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:12,361 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:12,461 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:12,562 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:12,662 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:12,762 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:12,862 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:12,962 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:13,063 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:13,163 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:13,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:13,363 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:13,464 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:13,564 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:13,664 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:13,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:13,865 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:13,965 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:14,065 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:14,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:14,265 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:14,366 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:14,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:14,566 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:14,666 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:14,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:14,866 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:14,967 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:15,067 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:15,167 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:15,267 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:15,367 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:15,468 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:15,568 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:15,668 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:15,768 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:15,868 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:15,969 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:16,069 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:16,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:16,269 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:16,369 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:16,470 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:16,570 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:16,670 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:16,770 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:16,870 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:16,970 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:17,071 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:17,171 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:17,271 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:17,371 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:17,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:17,572 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:17,672 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:17,772 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:17,873 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:17,973 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:18,073 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:18,173 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:18,273 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:18,374 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:18,474 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:18,574 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:18,674 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:18,774 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:18,874 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:18,975 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:19,075 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:19,175 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:19,275 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:19,375 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:19,476 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:19,576 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:19,676 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:19,776 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:19,876 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:19,977 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:20,077 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:20,177 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:20,277 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:20,377 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:20,478 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:20,578 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:20,678 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:20,778 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:20,878 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:20,979 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:21,079 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:21,179 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:21,279 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:21,379 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:21,480 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:21,580 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:21,680 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:21,780 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:21,880 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:21,981 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:22,081 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:22,181 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:22,281 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:22,381 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:22,482 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:22,582 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:22,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:22,782 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:22,882 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:22,983 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:23,083 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:23,183 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:23,283 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:23,383 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:23,484 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:23,538 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-24 12:16:23,584 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:23,639 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-24 12:16:23,684 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:23,784 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:23,840 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-24 12:16:23,884 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:23,985 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:24,085 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:24,141 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-24 12:16:24,185 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:24,285 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:24,385 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:24,486 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:24,586 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:24,642 DEBUG [RpcServer.default.FPBQ.Fifo.handler=1,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=88 2023-07-24 12:16:24,686 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:24,786 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:24,887 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:24,987 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:25,087 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:25,187 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:25,287 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:25,388 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:25,488 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:25,588 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:25,643 DEBUG [Listener at localhost/37279] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:25,645 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:25,646 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:25,646 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:25,646 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:25,648 ERROR [PEWorker-1] assignment.SplitTableRegionProcedure(347): Splitting 01a858500fdab441704f5de8ae140206, pid=95, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520 is currently in transition, give up at org.apache.hadoop.hbase.master.assignment.AssignmentManagerUtil.createUnassignProceduresForSplitOrMerge(AssignmentManagerUtil.java:120) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.createUnassignProcedures(SplitTableRegionProcedure.java:881) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:298) at org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure.executeFromState(SplitTableRegionProcedure.java:94) at org.apache.hadoop.hbase.procedure2.StateMachineProcedure.execute(StateMachineProcedure.java:188) at org.apache.hadoop.hbase.procedure2.Procedure.doExecute(Procedure.java:922) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1646) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:16:25,651 INFO [PEWorker-1] procedure2.ProcedureExecutor(1528): Rolled back pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea exec-time=7 msec 2023-07-24 12:16:25,688 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:25,788 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:25,889 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro(132): Slow down replica region open a bit 2023-07-24 12:16:25,989 INFO [StoreOpener-2e4f3d10799c74f83e99aa427a3ad520-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 2e4f3d10799c74f83e99aa427a3ad520 2023-07-24 12:16:25,993 DEBUG [StoreOpener-2e4f3d10799c74f83e99aa427a3ad520-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206/cf1 2023-07-24 12:16:25,993 DEBUG [StoreOpener-2e4f3d10799c74f83e99aa427a3ad520-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206/cf1 2023-07-24 12:16:25,994 INFO [StoreOpener-2e4f3d10799c74f83e99aa427a3ad520-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 2e4f3d10799c74f83e99aa427a3ad520 columnFamilyName cf1 2023-07-24 12:16:25,994 INFO [StoreOpener-2e4f3d10799c74f83e99aa427a3ad520-1] regionserver.HStore(310): Store=2e4f3d10799c74f83e99aa427a3ad520/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:25,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 2e4f3d10799c74f83e99aa427a3ad520; next sequenceid=5; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70551911, jitterRate=0.05130539834499359}}}, FlushLargeStoresPolicy{flushSizeLowerBound=-1} 2023-07-24 12:16:25,996 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 2e4f3d10799c74f83e99aa427a3ad520: 2023-07-24 12:16:25,996 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRollbackForSplitTableRegionWithReplica,,1690200922905_0001.2e4f3d10799c74f83e99aa427a3ad520., pid=94, masterSystemTime=1690200924261 2023-07-24 12:16:25,998 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRollbackForSplitTableRegionWithReplica,,1690200922905_0001.2e4f3d10799c74f83e99aa427a3ad520. 2023-07-24 12:16:25,998 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testRollbackForSplitTableRegionWithReplica,,1690200922905_0001.2e4f3d10799c74f83e99aa427a3ad520. 2023-07-24 12:16:25,998 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=93 updating hbase:meta row=2e4f3d10799c74f83e99aa427a3ad520, regionState=OPEN, openSeqNum=5, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:25,998 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690200985998"},{"qualifier":"server_0001","vlen":31,"tag":[],"timestamp":"1690200985998"},{"qualifier":"serverstartcode_0001","vlen":8,"tag":[],"timestamp":"1690200985998"},{"qualifier":"seqnumDuringOpen_0001","vlen":8,"tag":[],"timestamp":"1690200985998"}]},"ts":"1690200985998"} 2023-07-24 12:16:26,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=94, resume processing ppid=93 2023-07-24 12:16:26,001 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=94, ppid=93, state=SUCCESS; OpenRegionProcedure 2e4f3d10799c74f83e99aa427a3ad520, server=jenkins-hbase6.apache.org,34575,1690200894532 in 1 mins, 1.892 sec 2023-07-24 12:16:26,003 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=93, resume processing ppid=88 2023-07-24 12:16:26,003 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=93, ppid=88, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520, ASSIGN in 1 mins, 2.05 sec 2023-07-24 12:16:26,005 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica in 1 mins, 2.583 sec 2023-07-24 12:16:26,005 WARN [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] ipc.RpcServer(528): (responseTooSlow): {"call":"ModifyTable(org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest)","starttimems":"1690200923412","responsesize":"2","method":"ModifyTable","param":"TODO: class org.apache.hadoop.hbase.shaded.protobuf.generated.MasterProtos$ModifyTableRequest","processingtimems":62593,"client":"172.31.2.20:46316","queuetimems":0,"class":"HMaster"} 2023-07-24 12:16:26,005 DEBUG [RPCClient-NioEventLoopGroup-6-2] ipc.NettyRpcDuplexHandler(156): Unknown callId: 601, skipping over this response of 3 bytes 2023-07-24 12:16:27,896 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(335): procId=95 exception: org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520 is currently in transition, give up 2023-07-24 12:16:27,897 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:16:27,899 INFO [Listener at localhost/37279] client.HBaseAdmin$15(890): Started disable of testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:27,899 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$11(2418): Client=jenkins//172.31.2.20 disable testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:27,900 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=96, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:27,902 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-07-24 12:16:27,902 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200987902"}]},"ts":"1690200987902"} 2023-07-24 12:16:27,903 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLING in hbase:meta 2023-07-24 12:16:27,905 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRollbackForSplitTableRegionWithReplica to state=DISABLING 2023-07-24 12:16:27,905 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=01a858500fdab441704f5de8ae140206, UNASSIGN}, {pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520, UNASSIGN}] 2023-07-24 12:16:27,906 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=98, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520, UNASSIGN 2023-07-24 12:16:27,907 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=2e4f3d10799c74f83e99aa427a3ad520, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:27,907 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690200987907"},{"qualifier":"sn_0001","vlen":45,"tag":[],"timestamp":"1690200987907"},{"qualifier":"state_0001","vlen":7,"tag":[],"timestamp":"1690200987907"}]},"ts":"1690200987907"} 2023-07-24 12:16:27,908 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=99, ppid=98, state=RUNNABLE; CloseRegionProcedure 2e4f3d10799c74f83e99aa427a3ad520, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:27,910 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=97, ppid=96, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=01a858500fdab441704f5de8ae140206, UNASSIGN 2023-07-24 12:16:27,910 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=01a858500fdab441704f5de8ae140206, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:27,910 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690200987910"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200987910"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200987910"}]},"ts":"1690200987910"} 2023-07-24 12:16:27,911 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=100, ppid=97, state=RUNNABLE; CloseRegionProcedure 01a858500fdab441704f5de8ae140206, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:16:28,003 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-07-24 12:16:28,060 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 2e4f3d10799c74f83e99aa427a3ad520 2023-07-24 12:16:28,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 2e4f3d10799c74f83e99aa427a3ad520, disabling compactions & flushes 2023-07-24 12:16:28,061 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1690200922905_0001.2e4f3d10799c74f83e99aa427a3ad520. 2023-07-24 12:16:28,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1690200922905_0001.2e4f3d10799c74f83e99aa427a3ad520. 2023-07-24 12:16:28,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1690200922905_0001.2e4f3d10799c74f83e99aa427a3ad520. after waiting 0 ms 2023-07-24 12:16:28,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1690200922905_0001.2e4f3d10799c74f83e99aa427a3ad520. 2023-07-24 12:16:28,061 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:16:28,062 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1690200922905_0001.2e4f3d10799c74f83e99aa427a3ad520. 2023-07-24 12:16:28,062 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 2e4f3d10799c74f83e99aa427a3ad520: 2023-07-24 12:16:28,063 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 01a858500fdab441704f5de8ae140206 2023-07-24 12:16:28,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 01a858500fdab441704f5de8ae140206, disabling compactions & flushes 2023-07-24 12:16:28,063 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:16:28,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:16:28,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. after waiting 0 ms 2023-07-24 12:16:28,063 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:16:28,064 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 2e4f3d10799c74f83e99aa427a3ad520 2023-07-24 12:16:28,064 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=98 updating hbase:meta row=2e4f3d10799c74f83e99aa427a3ad520, regionState=CLOSED 2023-07-24 12:16:28,064 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690200988064"},{"qualifier":"state_0001","vlen":6,"tag":[],"timestamp":"1690200988064"}]},"ts":"1690200988064"} 2023-07-24 12:16:28,068 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206/recovered.edits/7.seqid, newMaxSeqId=7, maxSeqId=4 2023-07-24 12:16:28,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=99, resume processing ppid=98 2023-07-24 12:16:28,068 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=99, ppid=98, state=SUCCESS; CloseRegionProcedure 2e4f3d10799c74f83e99aa427a3ad520, server=jenkins-hbase6.apache.org,34575,1690200894532 in 158 msec 2023-07-24 12:16:28,068 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:16:28,069 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206. 2023-07-24 12:16:28,069 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 01a858500fdab441704f5de8ae140206: 2023-07-24 12:16:28,070 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=98, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520, UNASSIGN in 163 msec 2023-07-24 12:16:28,070 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 01a858500fdab441704f5de8ae140206 2023-07-24 12:16:28,070 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=97 updating hbase:meta row=01a858500fdab441704f5de8ae140206, regionState=CLOSED 2023-07-24 12:16:28,071 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.","families":{"info":[{"qualifier":"regioninfo","vlen":76,"tag":[],"timestamp":"1690200988070"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200988070"}]},"ts":"1690200988070"} 2023-07-24 12:16:28,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=100, resume processing ppid=97 2023-07-24 12:16:28,073 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=100, ppid=97, state=SUCCESS; CloseRegionProcedure 01a858500fdab441704f5de8ae140206, server=jenkins-hbase6.apache.org,36983,1690200894572 in 160 msec 2023-07-24 12:16:28,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=97, resume processing ppid=96 2023-07-24 12:16:28,074 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=97, ppid=96, state=SUCCESS; TransitRegionStateProcedure table=testRollbackForSplitTableRegionWithReplica, region=01a858500fdab441704f5de8ae140206, UNASSIGN in 168 msec 2023-07-24 12:16:28,075 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200988075"}]},"ts":"1690200988075"} 2023-07-24 12:16:28,076 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRollbackForSplitTableRegionWithReplica, state=DISABLED in hbase:meta 2023-07-24 12:16:28,078 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRollbackForSplitTableRegionWithReplica to state=DISABLED 2023-07-24 12:16:28,080 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica in 180 msec 2023-07-24 12:16:28,204 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=96 2023-07-24 12:16:28,204 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 96 completed 2023-07-24 12:16:28,205 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$5(2228): Client=jenkins//172.31.2.20 delete testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:28,206 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:28,207 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=101, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:28,208 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=101, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:28,208 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-24 12:16:28,212 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206 2023-07-24 12:16:28,213 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206/recovered.edits] 2023-07-24 12:16:28,218 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206/recovered.edits/7.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206/recovered.edits/7.seqid 2023-07-24 12:16:28,218 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRollbackForSplitTableRegionWithReplica/01a858500fdab441704f5de8ae140206 2023-07-24 12:16:28,218 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRollbackForSplitTableRegionWithReplica regions 2023-07-24 12:16:28,220 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=101, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:28,222 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testRollbackForSplitTableRegionWithReplica from hbase:meta 2023-07-24 12:16:28,224 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRollbackForSplitTableRegionWithReplica' descriptor. 2023-07-24 12:16:28,224 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=101, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:28,225 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRollbackForSplitTableRegionWithReplica' from region states. 2023-07-24 12:16:28,225 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200988225"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:28,225 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica,,1690200922905_0001.2e4f3d10799c74f83e99aa427a3ad520.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200988225"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:28,226 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 2 regions from META 2023-07-24 12:16:28,226 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 01a858500fdab441704f5de8ae140206, NAME => 'testRollbackForSplitTableRegionWithReplica,,1690200922905.01a858500fdab441704f5de8ae140206.', STARTKEY => '', ENDKEY => ''}, {ENCODED => 2e4f3d10799c74f83e99aa427a3ad520, NAME => 'testRollbackForSplitTableRegionWithReplica,,1690200922905_0001.2e4f3d10799c74f83e99aa427a3ad520.', STARTKEY => '', ENDKEY => '', REPLICA_ID => 1}] 2023-07-24 12:16:28,226 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRollbackForSplitTableRegionWithReplica' as deleted. 2023-07-24 12:16:28,226 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRollbackForSplitTableRegionWithReplica","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690200988226"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:28,228 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRollbackForSplitTableRegionWithReplica state from META 2023-07-24 12:16:28,229 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=101, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:28,230 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica in 24 msec 2023-07-24 12:16:28,309 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=101 2023-07-24 12:16:28,309 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRollbackForSplitTableRegionWithReplica, procId: 101 completed 2023-07-24 12:16:28,322 INFO [Listener at localhost/37279] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRollbackForSplitTableRegionWithReplica Thread=348 (was 415), OpenFileDescriptor=661 (was 877), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=180 (was 330), ProcessCount=168 (was 168), AvailableMemoryMB=1556 (was 1555) - AvailableMemoryMB LEAK? - 2023-07-24 12:16:28,336 INFO [Listener at localhost/37279] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=349, OpenFileDescriptor=661, MaxFileDescriptor=60000, SystemLoadAverage=180, ProcessCount=168, AvailableMemoryMB=1555 2023-07-24 12:16:28,337 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:16:28,338 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 12:16:28,339 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(492): Client=jenkins//172.31.2.20 set balanceSwitch=false 2023-07-24 12:16:28,340 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:28,341 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 12:16:28,342 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 12:16:28,343 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b 2023-07-24 12:16:28,344 DEBUG [HFileArchiver-10] backup.HFileArchiver(153): Directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b empty. 2023-07-24 12:16:28,344 DEBUG [HFileArchiver-10] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b 2023-07-24 12:16:28,344 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-07-24 12:16:28,353 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/.tabledesc/.tableinfo.0000000001 2023-07-24 12:16:28,354 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(7675): creating {ENCODED => 9b059f446a9fcafc1a8b94507646069b, NAME => 'testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testInvalidSplitKey', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp 2023-07-24 12:16:28,361 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:16:28,361 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1604): Closing 9b059f446a9fcafc1a8b94507646069b, disabling compactions & flushes 2023-07-24 12:16:28,361 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:28,361 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:28,361 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. after waiting 0 ms 2023-07-24 12:16:28,361 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:28,361 INFO [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:28,361 DEBUG [RegionOpenAndInit-testInvalidSplitKey-pool-0] regionserver.HRegion(1558): Region close journal for 9b059f446a9fcafc1a8b94507646069b: 2023-07-24 12:16:28,363 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 12:16:28,364 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1690200988364"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200988364"}]},"ts":"1690200988364"} 2023-07-24 12:16:28,365 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 12:16:28,366 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 12:16:28,366 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200988366"}]},"ts":"1690200988366"} 2023-07-24 12:16:28,367 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLING in hbase:meta 2023-07-24 12:16:28,371 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase6.apache.org=0} racks are {/default-rack=0} 2023-07-24 12:16:28,372 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 12:16:28,372 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 12:16:28,372 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 12:16:28,372 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 12:16:28,372 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=9b059f446a9fcafc1a8b94507646069b, ASSIGN}] 2023-07-24 12:16:28,373 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testInvalidSplitKey, region=9b059f446a9fcafc1a8b94507646069b, ASSIGN 2023-07-24 12:16:28,374 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=103, ppid=102, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testInvalidSplitKey, region=9b059f446a9fcafc1a8b94507646069b, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,34575,1690200894532; forceNewPlan=false, retain=false 2023-07-24 12:16:28,524 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 12:16:28,526 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=9b059f446a9fcafc1a8b94507646069b, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,526 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1690200988525"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200988525"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200988525"}]},"ts":"1690200988525"} 2023-07-24 12:16:28,528 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=104, ppid=103, state=RUNNABLE; OpenRegionProcedure 9b059f446a9fcafc1a8b94507646069b, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:28,682 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:28,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 9b059f446a9fcafc1a8b94507646069b, NAME => 'testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b.', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:16:28,683 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:16:28,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testInvalidSplitKey 9b059f446a9fcafc1a8b94507646069b 2023-07-24 12:16:28,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:16:28,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 9b059f446a9fcafc1a8b94507646069b 2023-07-24 12:16:28,683 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 9b059f446a9fcafc1a8b94507646069b 2023-07-24 12:16:28,684 INFO [StoreOpener-9b059f446a9fcafc1a8b94507646069b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 9b059f446a9fcafc1a8b94507646069b 2023-07-24 12:16:28,686 DEBUG [StoreOpener-9b059f446a9fcafc1a8b94507646069b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1 2023-07-24 12:16:28,686 DEBUG [StoreOpener-9b059f446a9fcafc1a8b94507646069b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1 2023-07-24 12:16:28,686 INFO [StoreOpener-9b059f446a9fcafc1a8b94507646069b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9b059f446a9fcafc1a8b94507646069b columnFamilyName cf1 2023-07-24 12:16:28,687 INFO [StoreOpener-9b059f446a9fcafc1a8b94507646069b-1] regionserver.HStore(310): Store=9b059f446a9fcafc1a8b94507646069b/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:28,687 INFO [StoreOpener-9b059f446a9fcafc1a8b94507646069b-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 9b059f446a9fcafc1a8b94507646069b 2023-07-24 12:16:28,688 DEBUG [StoreOpener-9b059f446a9fcafc1a8b94507646069b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2 2023-07-24 12:16:28,688 DEBUG [StoreOpener-9b059f446a9fcafc1a8b94507646069b-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2 2023-07-24 12:16:28,689 INFO [StoreOpener-9b059f446a9fcafc1a8b94507646069b-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 9b059f446a9fcafc1a8b94507646069b columnFamilyName cf2 2023-07-24 12:16:28,689 INFO [StoreOpener-9b059f446a9fcafc1a8b94507646069b-1] regionserver.HStore(310): Store=9b059f446a9fcafc1a8b94507646069b/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:28,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b 2023-07-24 12:16:28,690 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b 2023-07-24 12:16:28,692 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testInvalidSplitKey descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:16:28,693 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 9b059f446a9fcafc1a8b94507646069b 2023-07-24 12:16:28,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 12:16:28,695 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 9b059f446a9fcafc1a8b94507646069b; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=60019489, jitterRate=-0.10563991963863373}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:16:28,695 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 9b059f446a9fcafc1a8b94507646069b: 2023-07-24 12:16:28,696 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b., pid=104, masterSystemTime=1690200988679 2023-07-24 12:16:28,697 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:28,698 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:28,698 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=103 updating hbase:meta row=9b059f446a9fcafc1a8b94507646069b, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,698 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1690200988698"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200988698"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200988698"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200988698"}]},"ts":"1690200988698"} 2023-07-24 12:16:28,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=104, resume processing ppid=103 2023-07-24 12:16:28,701 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=104, ppid=103, state=SUCCESS; OpenRegionProcedure 9b059f446a9fcafc1a8b94507646069b, server=jenkins-hbase6.apache.org,34575,1690200894532 in 173 msec 2023-07-24 12:16:28,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=103, resume processing ppid=102 2023-07-24 12:16:28,703 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=103, ppid=102, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=9b059f446a9fcafc1a8b94507646069b, ASSIGN in 329 msec 2023-07-24 12:16:28,703 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 12:16:28,703 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200988703"}]},"ts":"1690200988703"} 2023-07-24 12:16:28,704 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=ENABLED in hbase:meta 2023-07-24 12:16:28,706 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=102, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testInvalidSplitKey execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 12:16:28,707 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey in 367 msec 2023-07-24 12:16:28,848 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:28,850 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-24 12:16:28,850 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-24 12:16:28,850 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:28,850 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-24 12:16:28,850 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-24 12:16:28,851 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,851 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:28,852 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:28,852 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:28,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:28,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:28,852 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:28,852 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:28,852 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:28,852 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:28,853 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,853 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:28,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:28,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:28,853 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,853 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:28,853 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:28,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:28,854 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:28,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:28,854 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:28,854 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:28,854 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:28,854 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:28,854 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:28,854 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:28,854 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:28,854 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:28,854 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:28,854 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:28,854 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:28,854 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:28,854 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:28,854 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:28,855 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:28,855 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:28,855 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:28,855 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:28,855 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:28,855 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:28,855 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:28,855 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. started... 2023-07-24 12:16:28,856 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9b059f446a9fcafc1a8b94507646069b 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 12:16:28,856 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,856 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,857 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,857 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:28,857 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,857 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:28,871 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf1/be515f604e0047c889c7bff9ebf4851a 2023-07-24 12:16:28,884 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf2/ca2e80aa9a8f41a2ba0e760f2bfb9927 2023-07-24 12:16:28,888 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf1/be515f604e0047c889c7bff9ebf4851a as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/be515f604e0047c889c7bff9ebf4851a 2023-07-24 12:16:28,892 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/be515f604e0047c889c7bff9ebf4851a, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:16:28,893 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf2/ca2e80aa9a8f41a2ba0e760f2bfb9927 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/ca2e80aa9a8f41a2ba0e760f2bfb9927 2023-07-24 12:16:28,897 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/ca2e80aa9a8f41a2ba0e760f2bfb9927, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:16:28,897 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 9b059f446a9fcafc1a8b94507646069b in 41ms, sequenceid=6, compaction requested=false 2023-07-24 12:16:28,898 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testInvalidSplitKey' 2023-07-24 12:16:28,898 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9b059f446a9fcafc1a8b94507646069b: 2023-07-24 12:16:28,898 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:28,898 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:28,898 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:28,898 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,898 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:28,898 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:28,901 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,901 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:28,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:28,901 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,901 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:28,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:28,901 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:28,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:28,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:28,902 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:28,903 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:28,903 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-24 12:16:28,903 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7b047f67[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:28,903 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-24 12:16:28,903 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,904 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,904 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,904 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,904 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,904 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:28,904 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:28,904 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:28,904 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,904 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:28,904 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:28,905 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:28,904 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,905 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:28,905 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:28,905 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:28,905 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:28,906 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:28,906 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:28,906 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,906 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:28,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:28,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:28,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:28,906 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:28,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:28,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:28,907 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:28,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:28,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:28,908 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,909 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:28,909 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-24 12:16:28,909 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-24 12:16:28,909 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:28,909 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:28,909 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-24 12:16:28,909 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:28,910 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:28,910 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:28,910 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:28,910 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:28,910 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:28,910 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,911 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:28,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,910 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,910 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,911 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:28,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,911 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:28,910 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:28,910 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:28,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:28,911 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:28,911 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:28,911 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:28,911 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:28,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:28,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:28,912 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:28,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:28,912 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:28,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:28,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:28,913 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:28,914 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:28,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:28,914 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:28,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:28,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:28,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:28,918 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:28,918 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:28,918 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:28,918 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:28,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:28,918 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:28,918 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:28,918 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:28,918 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:28,918 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:28,918 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:28,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:28,918 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:28,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:28,918 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:28,918 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-24 12:16:28,919 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:28,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:28,918 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:28,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:28,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:28,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:28,919 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,919 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:28,919 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:28,919 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:28,919 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:28,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:28,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:28,919 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:28,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:28,919 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:28,919 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,019 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:29,020 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:29,032 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:29,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:29,033 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-24 12:16:29,034 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-24 12:16:29,034 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,034 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-24 12:16:29,034 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-24 12:16:29,034 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,034 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,035 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,035 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,035 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,035 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,035 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,035 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,036 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,036 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:29,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,036 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,036 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,036 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,036 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,037 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,037 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,037 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,037 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,037 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,037 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,037 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,037 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,037 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,037 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,037 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,037 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,037 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,037 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,037 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,037 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,037 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,038 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,037 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,038 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,038 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:29,038 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:29,038 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,038 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,038 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,038 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. started... 2023-07-24 12:16:29,038 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:29,038 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9b059f446a9fcafc1a8b94507646069b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:29,039 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,046 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,046 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,046 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,046 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,046 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,051 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf1/255a2c43d64941fab9f006f2f293d6d9 2023-07-24 12:16:29,064 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf2/32941b45e4424c3f88c151cafd833167 2023-07-24 12:16:29,068 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf1/255a2c43d64941fab9f006f2f293d6d9 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/255a2c43d64941fab9f006f2f293d6d9 2023-07-24 12:16:29,073 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/255a2c43d64941fab9f006f2f293d6d9, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:16:29,073 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf2/32941b45e4424c3f88c151cafd833167 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/32941b45e4424c3f88c151cafd833167 2023-07-24 12:16:29,077 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/32941b45e4424c3f88c151cafd833167, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:16:29,078 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9b059f446a9fcafc1a8b94507646069b in 40ms, sequenceid=19, compaction requested=false 2023-07-24 12:16:29,078 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9b059f446a9fcafc1a8b94507646069b: 2023-07-24 12:16:29,078 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:29,078 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:29,078 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:29,078 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,078 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,078 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,080 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,080 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,080 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,080 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,081 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,081 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,082 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,083 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-24 12:16:29,083 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4380f235[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:29,083 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-24 12:16:29,083 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,084 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,084 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,084 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,084 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,084 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,084 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,084 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:29,084 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,084 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:29,084 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,084 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,085 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,085 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:29,084 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,085 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,085 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:29,086 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,086 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,086 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,086 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,086 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,087 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,087 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,087 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,088 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,088 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,088 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:29,091 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-24 12:16:29,091 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-24 12:16:29,091 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:29,091 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:29,092 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-24 12:16:29,092 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:29,093 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,093 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,093 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,093 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,093 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,093 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,093 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,093 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,094 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,094 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,095 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,095 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,097 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,097 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,098 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,101 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,101 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-24 12:16:29,101 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:29,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,101 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:29,102 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:29,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,102 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,102 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,102 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,102 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,102 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,202 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:29,203 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:29,214 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:29,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:29,216 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-24 12:16:29,216 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-24 12:16:29,216 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,216 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-24 12:16:29,216 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-24 12:16:29,216 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,216 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,219 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,219 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,219 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,219 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,219 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,219 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,219 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:29,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,219 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,220 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,220 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,220 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,220 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,221 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,221 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,221 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,221 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,221 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,221 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,221 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,221 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,221 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,221 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,221 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,221 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,221 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,221 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,221 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,221 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,221 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,221 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,222 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:29,222 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:29,222 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:29,222 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. started... 2023-07-24 12:16:29,222 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9b059f446a9fcafc1a8b94507646069b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:29,223 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,223 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,223 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,223 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,223 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,223 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,234 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf1/e6eb4ae15fb94c4e810ba265ba5dc8c1 2023-07-24 12:16:29,249 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf2/c237865b907f41eb906ff5af6a4a1d6f 2023-07-24 12:16:29,253 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf1/e6eb4ae15fb94c4e810ba265ba5dc8c1 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/e6eb4ae15fb94c4e810ba265ba5dc8c1 2023-07-24 12:16:29,257 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/e6eb4ae15fb94c4e810ba265ba5dc8c1, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:16:29,258 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf2/c237865b907f41eb906ff5af6a4a1d6f as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/c237865b907f41eb906ff5af6a4a1d6f 2023-07-24 12:16:29,262 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/c237865b907f41eb906ff5af6a4a1d6f, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:16:29,262 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9b059f446a9fcafc1a8b94507646069b in 40ms, sequenceid=32, compaction requested=true 2023-07-24 12:16:29,262 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9b059f446a9fcafc1a8b94507646069b: 2023-07-24 12:16:29,263 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:29,263 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:29,263 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:29,263 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,263 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,263 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,264 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,264 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,265 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,265 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,267 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-24 12:16:29,267 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@315e3aef[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:29,267 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-24 12:16:29,267 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,268 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,268 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,268 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,268 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,268 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,268 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,268 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,268 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,268 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:29,268 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,268 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:29,268 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:29,268 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,269 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:29,271 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,271 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,271 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,272 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,272 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,272 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,272 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,272 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,272 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,272 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:29,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-24 12:16:29,275 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-24 12:16:29,275 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:29,275 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:29,275 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-24 12:16:29,275 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:29,277 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,277 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,277 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,277 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,277 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,277 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,277 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,277 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,277 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,277 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,278 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,278 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,278 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,278 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,280 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,280 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,280 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,286 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,287 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,287 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,287 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,287 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,287 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,287 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,287 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,287 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-24 12:16:29,287 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:29,287 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,287 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,287 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:29,287 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,287 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,287 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,287 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:29,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,288 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,288 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,288 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,288 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,288 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,288 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,288 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,388 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:29,389 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:29,400 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:29,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:29,402 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-24 12:16:29,402 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-24 12:16:29,402 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,402 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-24 12:16:29,402 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-24 12:16:29,403 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,403 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,404 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,404 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,404 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,404 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,404 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,404 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,404 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,404 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:29,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,405 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,405 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,405 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,405 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,405 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,405 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,406 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,406 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,406 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,406 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,406 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,406 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,406 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,406 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,406 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. started... 2023-07-24 12:16:29,406 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,407 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9b059f446a9fcafc1a8b94507646069b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:29,408 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,408 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,408 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,408 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,408 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,408 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,416 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf1/a8a7471aa54b41bfb0a21ca4283b3e03 2023-07-24 12:16:29,430 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf2/6525eca8c2e04a2f861c9d4fc9c1d1dd 2023-07-24 12:16:29,434 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf1/a8a7471aa54b41bfb0a21ca4283b3e03 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/a8a7471aa54b41bfb0a21ca4283b3e03 2023-07-24 12:16:29,438 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/a8a7471aa54b41bfb0a21ca4283b3e03, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:16:29,439 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf2/6525eca8c2e04a2f861c9d4fc9c1d1dd as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/6525eca8c2e04a2f861c9d4fc9c1d1dd 2023-07-24 12:16:29,444 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/6525eca8c2e04a2f861c9d4fc9c1d1dd, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:16:29,445 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9b059f446a9fcafc1a8b94507646069b in 38ms, sequenceid=45, compaction requested=true 2023-07-24 12:16:29,445 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9b059f446a9fcafc1a8b94507646069b: 2023-07-24 12:16:29,445 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:29,445 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:29,445 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:29,445 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,445 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,445 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,447 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,447 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,447 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,447 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-24 12:16:29,449 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@5339de27[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:29,449 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-24 12:16:29,449 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,451 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,451 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,451 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,452 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,452 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,452 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:29,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,452 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,452 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:29,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,452 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,452 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,452 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,452 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,452 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,452 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:29,452 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:29,453 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,453 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,453 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,454 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,454 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,454 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,455 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,455 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,455 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,455 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,455 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,456 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,457 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,458 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,459 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:29,459 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-24 12:16:29,459 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-24 12:16:29,459 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:29,459 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:29,459 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-24 12:16:29,459 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:29,461 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,461 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,461 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,461 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,461 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,462 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,461 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,461 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,461 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,462 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,462 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,462 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,465 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,465 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,465 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,466 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,467 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,467 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,468 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,468 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,468 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,470 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,470 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,470 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,470 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,471 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,471 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,471 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-24 12:16:29,471 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:29,471 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,471 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,471 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,471 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:29,471 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,471 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:29,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,471 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,471 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,472 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,472 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,472 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,472 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,472 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,472 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,571 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:29,572 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:29,583 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:29,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:29,585 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-24 12:16:29,585 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-24 12:16:29,585 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,585 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-24 12:16:29,585 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-24 12:16:29,585 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,585 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,586 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,586 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,587 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,586 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,587 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,587 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,587 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:29,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,587 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,587 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,588 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,588 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,588 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,588 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,588 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,588 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,588 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,588 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,588 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,588 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,588 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,588 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,589 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,588 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,589 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:29,589 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:29,589 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,589 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,589 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,589 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:29,589 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,589 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. started... 2023-07-24 12:16:29,589 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,589 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,589 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,589 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,589 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9b059f446a9fcafc1a8b94507646069b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:29,589 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,590 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,592 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,592 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,592 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,592 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,592 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,597 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf1/c00557c7a05d4cc3a2399c9651a31814 2023-07-24 12:16:29,608 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf2/3b69e3ab4b794a5ebb9cea0a231af818 2023-07-24 12:16:29,612 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf1/c00557c7a05d4cc3a2399c9651a31814 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/c00557c7a05d4cc3a2399c9651a31814 2023-07-24 12:16:29,615 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/c00557c7a05d4cc3a2399c9651a31814, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:16:29,616 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf2/3b69e3ab4b794a5ebb9cea0a231af818 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/3b69e3ab4b794a5ebb9cea0a231af818 2023-07-24 12:16:29,619 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/3b69e3ab4b794a5ebb9cea0a231af818, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:16:29,620 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9b059f446a9fcafc1a8b94507646069b in 31ms, sequenceid=58, compaction requested=true 2023-07-24 12:16:29,620 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9b059f446a9fcafc1a8b94507646069b: 2023-07-24 12:16:29,620 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:29,620 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:29,621 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:29,621 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,621 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,621 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,622 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,622 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,622 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,622 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,624 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-24 12:16:29,624 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@b9e6c60[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:29,624 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-24 12:16:29,624 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,625 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,625 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,625 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,625 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,625 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,625 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:29,625 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,625 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,626 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,626 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:29,625 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,626 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,626 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,626 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:29,626 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:29,630 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,630 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,631 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,631 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,631 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,633 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,633 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,633 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,633 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,633 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,634 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,633 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,634 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,635 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,636 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:29,636 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-24 12:16:29,636 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-24 12:16:29,636 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:29,636 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:29,636 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-24 12:16:29,636 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:29,638 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,638 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,638 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,638 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,638 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,638 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,638 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,638 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,638 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,638 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,639 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,639 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,639 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,639 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,640 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,641 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,641 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,641 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,642 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,642 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,644 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,644 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,644 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,644 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,644 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,644 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-24 12:16:29,644 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:29,644 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,644 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,644 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,644 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,645 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:29,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,644 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,645 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,645 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:29,644 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,644 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,645 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,645 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,645 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,645 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,645 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,645 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,645 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,745 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:29,746 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:29,757 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:29,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testInvalidSplitKey was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:29,758 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testInvalidSplitKey 2023-07-24 12:16:29,759 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testInvalidSplitKey' 2023-07-24 12:16:29,759 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,759 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testInvalidSplitKey' starting 'acquire' 2023-07-24 12:16:29,759 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testInvalidSplitKey', kicking off acquire phase on members. 2023-07-24 12:16:29,763 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,763 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,765 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,765 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,765 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,765 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,765 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,765 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,765 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,765 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,765 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,766 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,766 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:29,766 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,766 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,766 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,766 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testInvalidSplitKey 2023-07-24 12:16:29,766 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,766 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,766 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,766 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,766 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testInvalidSplitKey is still present, removing 2023-07-24 12:16:29,766 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,766 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testInvalidSplitKey 2023-07-24 12:16:29,766 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,766 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,767 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,767 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,767 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,767 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,767 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testInvalidSplitKey' with timeout 60000ms 2023-07-24 12:16:29,767 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,767 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:29,767 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testInvalidSplitKey' starting 'acquire' stage 2023-07-24 12:16:29,767 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:29,767 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:29,767 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:29,767 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. started... 2023-07-24 12:16:29,767 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 9b059f446a9fcafc1a8b94507646069b 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:29,768 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,769 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,769 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,769 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,769 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,769 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,777 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf1/88cfeba9bcc24fb682c6d349224daab0 2023-07-24 12:16:29,788 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf2/108d4284028a45dba3e9946bad70d0d2 2023-07-24 12:16:29,792 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf1/88cfeba9bcc24fb682c6d349224daab0 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/88cfeba9bcc24fb682c6d349224daab0 2023-07-24 12:16:29,796 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/88cfeba9bcc24fb682c6d349224daab0, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:16:29,796 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf2/108d4284028a45dba3e9946bad70d0d2 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/108d4284028a45dba3e9946bad70d0d2 2023-07-24 12:16:29,800 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/108d4284028a45dba3e9946bad70d0d2, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:16:29,801 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 9b059f446a9fcafc1a8b94507646069b in 34ms, sequenceid=71, compaction requested=true 2023-07-24 12:16:29,801 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 9b059f446a9fcafc1a8b94507646069b: 2023-07-24 12:16:29,801 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:29,801 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:29,801 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:29,801 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,801 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testInvalidSplitKey' locally acquired 2023-07-24 12:16:29,801 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testInvalidSplitKey) in zk 2023-07-24 12:16:29,802 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,802 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,803 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,803 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testInvalidSplitKey' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:29,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,803 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,804 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testInvalidSplitKey' on coordinator 2023-07-24 12:16:29,805 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4fcb267c[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:29,805 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testInvalidSplitKey' starting 'in-barrier' execution. 2023-07-24 12:16:29,805 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,807 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,807 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,807 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,807 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,807 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,807 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:29,807 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,807 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,807 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,807 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,807 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:29,807 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,807 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:29,807 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testInvalidSplitKey' received 'reached' from coordinator. 2023-07-24 12:16:29,807 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testInvalidSplitKey' locally completed 2023-07-24 12:16:29,807 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testInvalidSplitKey' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:29,808 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,808 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,809 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,809 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,809 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,809 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,810 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,810 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testInvalidSplitKey' has notified controller of completion 2023-07-24 12:16:29,810 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,810 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testInvalidSplitKey' completed. 2023-07-24 12:16:29,810 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,811 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,812 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,813 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testInvalidSplitKey' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:29,813 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testInvalidSplitKey', counting down latch. Waiting for 0 more 2023-07-24 12:16:29,813 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testInvalidSplitKey' execution completed 2023-07-24 12:16:29,813 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:29,813 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:29,813 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testInvalidSplitKey 2023-07-24 12:16:29,813 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testInvalidSplitKeyincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:29,815 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,815 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,815 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,816 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,816 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,816 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,816 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:29,816 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,816 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:29,816 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,817 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:29,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,817 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,817 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:29,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,819 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,819 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:29,819 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testInvalidSplitKey 2023-07-24 12:16:29,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,820 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,821 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testInvalidSplitKey 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testInvalidSplitKey 2023-07-24 12:16:29,822 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,821 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testInvalidSplitKey 2023-07-24 12:16:29,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testInvalidSplitKey' 2023-07-24 12:16:29,822 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:29,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,822 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:29,822 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testInvalidSplitKey'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:29,822 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:29,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,823 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:29,823 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:29,923 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:29,923 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:29,930 INFO [Listener at localhost/37279] assignment.SplitTableRegionProcedure(206): testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. splitKey isn't explicitly specified, will try to find a best split key from RS jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,931 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.StoreUtils(129): cannot split hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/255a2c43d64941fab9f006f2f293d6d9 because midkey is the same as first or last row 2023-07-24 12:16:29,931 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.StoreUtils(129): cannot split hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/32941b45e4424c3f88c151cafd833167 because midkey is the same as first or last row 2023-07-24 12:16:29,932 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.HRegion(2745): Flushing 9b059f446a9fcafc1a8b94507646069b 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-24 12:16:29,940 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf1/8268ebfb8406435487e970d1ea62711c 2023-07-24 12:16:29,950 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf2/2edae98a56904c78be29c1904dc0438c 2023-07-24 12:16:29,954 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf1/8268ebfb8406435487e970d1ea62711c as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/8268ebfb8406435487e970d1ea62711c 2023-07-24 12:16:29,957 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/8268ebfb8406435487e970d1ea62711c, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 12:16:29,958 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/.tmp/cf2/2edae98a56904c78be29c1904dc0438c as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/2edae98a56904c78be29c1904dc0438c 2023-07-24 12:16:29,962 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/2edae98a56904c78be29c1904dc0438c, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 12:16:29,962 INFO [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 9b059f446a9fcafc1a8b94507646069b in 30ms, sequenceid=82, compaction requested=true 2023-07-24 12:16:29,963 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.HRegion(2446): Flush status journal for 9b059f446a9fcafc1a8b94507646069b: 2023-07-24 12:16:29,963 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.StoreUtils(129): cannot split hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/255a2c43d64941fab9f006f2f293d6d9 because midkey is the same as first or last row 2023-07-24 12:16:29,964 DEBUG [RpcServer.priority.RWQ.Fifo.read.handler=1,queue=1,port=34575] regionserver.StoreUtils(129): cannot split hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/32941b45e4424c3f88c151cafd833167 because midkey is the same as first or last row 2023-07-24 12:16:29,964 DEBUG [Listener at localhost/37279] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,964 DEBUG [Listener at localhost/37279] assignment.TestSplitTableRegionProcedure(438): Expected Split procedure construction failure: Region not splittable because bestSplitPoint = null, maybe table is too small for auto split. For force split, try specifying split row 2023-07-24 12:16:29,964 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:16:29,965 INFO [Listener at localhost/37279] client.HBaseAdmin$15(890): Started disable of testInvalidSplitKey 2023-07-24 12:16:29,966 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$11(2418): Client=jenkins//172.31.2.20 disable testInvalidSplitKey 2023-07-24 12:16:29,966 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=105, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:29,968 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-07-24 12:16:29,969 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200989969"}]},"ts":"1690200989969"} 2023-07-24 12:16:29,970 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLING in hbase:meta 2023-07-24 12:16:29,971 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testInvalidSplitKey to state=DISABLING 2023-07-24 12:16:29,972 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=9b059f446a9fcafc1a8b94507646069b, UNASSIGN}] 2023-07-24 12:16:29,974 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=106, ppid=105, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testInvalidSplitKey, region=9b059f446a9fcafc1a8b94507646069b, UNASSIGN 2023-07-24 12:16:29,975 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=9b059f446a9fcafc1a8b94507646069b, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:29,975 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1690200989974"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200989974"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200989974"}]},"ts":"1690200989974"} 2023-07-24 12:16:29,976 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=107, ppid=106, state=RUNNABLE; CloseRegionProcedure 9b059f446a9fcafc1a8b94507646069b, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:30,069 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-07-24 12:16:30,127 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 9b059f446a9fcafc1a8b94507646069b 2023-07-24 12:16:30,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 9b059f446a9fcafc1a8b94507646069b, disabling compactions & flushes 2023-07-24 12:16:30,128 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:30,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:30,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. after waiting 0 ms 2023-07-24 12:16:30,128 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:30,132 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-24 12:16:30,132 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:16:30,132 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b. 2023-07-24 12:16:30,132 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 9b059f446a9fcafc1a8b94507646069b: 2023-07-24 12:16:30,134 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 9b059f446a9fcafc1a8b94507646069b 2023-07-24 12:16:30,134 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=106 updating hbase:meta row=9b059f446a9fcafc1a8b94507646069b, regionState=CLOSED 2023-07-24 12:16:30,134 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b.","families":{"info":[{"qualifier":"regioninfo","vlen":53,"tag":[],"timestamp":"1690200990134"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200990134"}]},"ts":"1690200990134"} 2023-07-24 12:16:30,136 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=107, resume processing ppid=106 2023-07-24 12:16:30,136 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=107, ppid=106, state=SUCCESS; CloseRegionProcedure 9b059f446a9fcafc1a8b94507646069b, server=jenkins-hbase6.apache.org,34575,1690200894532 in 159 msec 2023-07-24 12:16:30,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=106, resume processing ppid=105 2023-07-24 12:16:30,138 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=106, ppid=105, state=SUCCESS; TransitRegionStateProcedure table=testInvalidSplitKey, region=9b059f446a9fcafc1a8b94507646069b, UNASSIGN in 165 msec 2023-07-24 12:16:30,138 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200990138"}]},"ts":"1690200990138"} 2023-07-24 12:16:30,139 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testInvalidSplitKey, state=DISABLED in hbase:meta 2023-07-24 12:16:30,142 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testInvalidSplitKey to state=DISABLED 2023-07-24 12:16:30,144 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey in 177 msec 2023-07-24 12:16:30,270 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=105 2023-07-24 12:16:30,270 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testInvalidSplitKey, procId: 105 completed 2023-07-24 12:16:30,271 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$5(2228): Client=jenkins//172.31.2.20 delete testInvalidSplitKey 2023-07-24 12:16:30,272 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:30,273 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=108, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:30,274 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=108, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:30,274 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-07-24 12:16:30,277 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b 2023-07-24 12:16:30,278 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/recovered.edits] 2023-07-24 12:16:30,282 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/255a2c43d64941fab9f006f2f293d6d9 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/255a2c43d64941fab9f006f2f293d6d9 2023-07-24 12:16:30,283 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/8268ebfb8406435487e970d1ea62711c to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/8268ebfb8406435487e970d1ea62711c 2023-07-24 12:16:30,284 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/88cfeba9bcc24fb682c6d349224daab0 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/88cfeba9bcc24fb682c6d349224daab0 2023-07-24 12:16:30,285 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/a8a7471aa54b41bfb0a21ca4283b3e03 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/a8a7471aa54b41bfb0a21ca4283b3e03 2023-07-24 12:16:30,286 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/be515f604e0047c889c7bff9ebf4851a to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/be515f604e0047c889c7bff9ebf4851a 2023-07-24 12:16:30,287 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/c00557c7a05d4cc3a2399c9651a31814 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/c00557c7a05d4cc3a2399c9651a31814 2023-07-24 12:16:30,288 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/e6eb4ae15fb94c4e810ba265ba5dc8c1 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf1/e6eb4ae15fb94c4e810ba265ba5dc8c1 2023-07-24 12:16:30,290 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/108d4284028a45dba3e9946bad70d0d2 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/108d4284028a45dba3e9946bad70d0d2 2023-07-24 12:16:30,291 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/2edae98a56904c78be29c1904dc0438c to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/2edae98a56904c78be29c1904dc0438c 2023-07-24 12:16:30,292 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/32941b45e4424c3f88c151cafd833167 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/32941b45e4424c3f88c151cafd833167 2023-07-24 12:16:30,293 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/3b69e3ab4b794a5ebb9cea0a231af818 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/3b69e3ab4b794a5ebb9cea0a231af818 2023-07-24 12:16:30,294 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/6525eca8c2e04a2f861c9d4fc9c1d1dd to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/6525eca8c2e04a2f861c9d4fc9c1d1dd 2023-07-24 12:16:30,295 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/c237865b907f41eb906ff5af6a4a1d6f to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/c237865b907f41eb906ff5af6a4a1d6f 2023-07-24 12:16:30,296 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/ca2e80aa9a8f41a2ba0e760f2bfb9927 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/cf2/ca2e80aa9a8f41a2ba0e760f2bfb9927 2023-07-24 12:16:30,298 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/recovered.edits/85.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b/recovered.edits/85.seqid 2023-07-24 12:16:30,299 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testInvalidSplitKey/9b059f446a9fcafc1a8b94507646069b 2023-07-24 12:16:30,299 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testInvalidSplitKey regions 2023-07-24 12:16:30,300 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=108, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:30,302 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 1 rows of testInvalidSplitKey from hbase:meta 2023-07-24 12:16:30,305 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testInvalidSplitKey' descriptor. 2023-07-24 12:16:30,306 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=108, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:30,306 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testInvalidSplitKey' from region states. 2023-07-24 12:16:30,306 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200990306"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:30,307 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 1 regions from META 2023-07-24 12:16:30,307 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 9b059f446a9fcafc1a8b94507646069b, NAME => 'testInvalidSplitKey,,1690200988339.9b059f446a9fcafc1a8b94507646069b.', STARTKEY => '', ENDKEY => ''}] 2023-07-24 12:16:30,307 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testInvalidSplitKey' as deleted. 2023-07-24 12:16:30,307 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testInvalidSplitKey","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690200990307"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:30,309 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testInvalidSplitKey state from META 2023-07-24 12:16:30,310 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=108, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:30,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey in 39 msec 2023-07-24 12:16:30,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=108 2023-07-24 12:16:30,375 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testInvalidSplitKey, procId: 108 completed 2023-07-24 12:16:30,390 INFO [Listener at localhost/37279] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testInvalidSplitKey Thread=371 (was 349) - Thread LEAK? -, OpenFileDescriptor=750 (was 661) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=182 (was 180) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=1546 (was 1555) 2023-07-24 12:16:30,403 INFO [Listener at localhost/37279] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=371, OpenFileDescriptor=750, MaxFileDescriptor=60000, SystemLoadAverage=182, ProcessCount=168, AvailableMemoryMB=1545 2023-07-24 12:16:30,404 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:16:30,405 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 12:16:30,406 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(492): Client=jenkins//172.31.2.20 set balanceSwitch=false 2023-07-24 12:16:30,407 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:30,408 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 12:16:30,409 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 12:16:30,410 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:30,411 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141 empty. 2023-07-24 12:16:30,411 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:30,411 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-07-24 12:16:30,420 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/.tabledesc/.tableinfo.0000000001 2023-07-24 12:16:30,421 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(7675): creating {ENCODED => d8d7c0b2b9c0adab4dd247a24a5b7141, NAME => 'testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testRecoveryAndDoubleExecution', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp 2023-07-24 12:16:30,428 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:16:30,428 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1604): Closing d8d7c0b2b9c0adab4dd247a24a5b7141, disabling compactions & flushes 2023-07-24 12:16:30,428 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:30,428 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:30,428 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. after waiting 0 ms 2023-07-24 12:16:30,428 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:30,428 INFO [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:30,428 DEBUG [RegionOpenAndInit-testRecoveryAndDoubleExecution-pool-0] regionserver.HRegion(1558): Region close journal for d8d7c0b2b9c0adab4dd247a24a5b7141: 2023-07-24 12:16:30,429 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 12:16:30,430 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200990430"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200990430"}]},"ts":"1690200990430"} 2023-07-24 12:16:30,431 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 12:16:30,432 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 12:16:30,432 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200990432"}]},"ts":"1690200990432"} 2023-07-24 12:16:30,433 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLING in hbase:meta 2023-07-24 12:16:30,442 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase6.apache.org=0} racks are {/default-rack=0} 2023-07-24 12:16:30,442 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 12:16:30,442 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 12:16:30,442 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 12:16:30,443 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 12:16:30,443 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, ASSIGN}] 2023-07-24 12:16:30,444 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, ASSIGN 2023-07-24 12:16:30,444 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=110, ppid=109, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,34575,1690200894532; forceNewPlan=false, retain=false 2023-07-24 12:16:30,594 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 12:16:30,596 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,596 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200990596"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200990596"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200990596"}]},"ts":"1690200990596"} 2023-07-24 12:16:30,597 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=111, ppid=110, state=RUNNABLE; OpenRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:30,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:30,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => d8d7c0b2b9c0adab4dd247a24a5b7141, NAME => 'testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141.', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:16:30,752 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:16:30,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:30,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:16:30,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:30,752 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:30,754 INFO [StoreOpener-d8d7c0b2b9c0adab4dd247a24a5b7141-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:30,755 DEBUG [StoreOpener-d8d7c0b2b9c0adab4dd247a24a5b7141-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1 2023-07-24 12:16:30,755 DEBUG [StoreOpener-d8d7c0b2b9c0adab4dd247a24a5b7141-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1 2023-07-24 12:16:30,755 INFO [StoreOpener-d8d7c0b2b9c0adab4dd247a24a5b7141-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d8d7c0b2b9c0adab4dd247a24a5b7141 columnFamilyName cf1 2023-07-24 12:16:30,756 INFO [StoreOpener-d8d7c0b2b9c0adab4dd247a24a5b7141-1] regionserver.HStore(310): Store=d8d7c0b2b9c0adab4dd247a24a5b7141/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:30,756 INFO [StoreOpener-d8d7c0b2b9c0adab4dd247a24a5b7141-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:30,757 DEBUG [StoreOpener-d8d7c0b2b9c0adab4dd247a24a5b7141-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2 2023-07-24 12:16:30,757 DEBUG [StoreOpener-d8d7c0b2b9c0adab4dd247a24a5b7141-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2 2023-07-24 12:16:30,757 INFO [StoreOpener-d8d7c0b2b9c0adab4dd247a24a5b7141-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region d8d7c0b2b9c0adab4dd247a24a5b7141 columnFamilyName cf2 2023-07-24 12:16:30,758 INFO [StoreOpener-d8d7c0b2b9c0adab4dd247a24a5b7141-1] regionserver.HStore(310): Store=d8d7c0b2b9c0adab4dd247a24a5b7141/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:30,758 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:30,759 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:30,760 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:16:30,762 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:30,763 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 12:16:30,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened d8d7c0b2b9c0adab4dd247a24a5b7141; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=70816808, jitterRate=0.055252671241760254}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:16:30,764 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for d8d7c0b2b9c0adab4dd247a24a5b7141: 2023-07-24 12:16:30,764 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141., pid=111, masterSystemTime=1690200990749 2023-07-24 12:16:30,766 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:30,766 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:30,766 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=110 updating hbase:meta row=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,766 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200990766"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200990766"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200990766"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200990766"}]},"ts":"1690200990766"} 2023-07-24 12:16:30,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=111, resume processing ppid=110 2023-07-24 12:16:30,768 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=111, ppid=110, state=SUCCESS; OpenRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 in 170 msec 2023-07-24 12:16:30,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=110, resume processing ppid=109 2023-07-24 12:16:30,770 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=110, ppid=109, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, ASSIGN in 326 msec 2023-07-24 12:16:30,770 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 12:16:30,771 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200990771"}]},"ts":"1690200990771"} 2023-07-24 12:16:30,772 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=ENABLED in hbase:meta 2023-07-24 12:16:30,774 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=109, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testRecoveryAndDoubleExecution execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 12:16:30,775 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution in 368 msec 2023-07-24 12:16:30,914 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:30,916 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-24 12:16:30,916 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-24 12:16:30,916 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:30,916 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-24 12:16:30,916 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 12:16:30,916 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,916 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,917 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:30,917 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:30,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:30,917 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:30,917 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:30,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:30,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:30,918 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,918 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:30,918 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:30,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:30,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,918 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,918 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,918 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:30,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:30,919 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:30,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:30,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,919 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:30,919 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:30,919 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:30,919 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:30,919 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:30,919 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:30,919 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:30,919 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:30,919 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:30,919 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:30,919 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:30,919 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:30,919 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:30,919 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:30,919 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:30,919 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:30,919 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:30,919 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:30,920 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:30,920 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:30,920 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:30,920 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. started... 2023-07-24 12:16:30,921 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d8d7c0b2b9c0adab4dd247a24a5b7141 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 12:16:30,922 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,923 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,923 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,923 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:30,923 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,923 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:30,930 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf 2023-07-24 12:16:30,941 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf2/99f25d22508e4e5c90b523abeff3417b 2023-07-24 12:16:30,944 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf 2023-07-24 12:16:30,947 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:16:30,948 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf2/99f25d22508e4e5c90b523abeff3417b as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/99f25d22508e4e5c90b523abeff3417b 2023-07-24 12:16:30,951 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/99f25d22508e4e5c90b523abeff3417b, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:16:30,952 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for d8d7c0b2b9c0adab4dd247a24a5b7141 in 31ms, sequenceid=6, compaction requested=false 2023-07-24 12:16:30,952 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testRecoveryAndDoubleExecution' 2023-07-24 12:16:30,952 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d8d7c0b2b9c0adab4dd247a24a5b7141: 2023-07-24 12:16:30,952 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:30,953 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:30,953 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:30,953 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,953 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:30,953 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:30,954 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,954 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:30,954 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:30,954 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,954 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:30,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:30,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:30,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:30,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,955 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:30,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:30,956 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:30,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-24 12:16:30,956 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@f02e428[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:30,956 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 12:16:30,956 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,957 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,957 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,957 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,957 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,957 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:30,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,957 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:30,958 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:30,958 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:30,958 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:30,957 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,958 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:30,958 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:30,958 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:30,958 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:30,958 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,958 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:30,959 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:30,959 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,959 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:30,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:30,959 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:30,959 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:30,960 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:30,960 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:30,960 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:30,960 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:30,960 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:30,960 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:30,960 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:30,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:30,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:30,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,961 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:30,962 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:30,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:30,963 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:30,963 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 12:16:30,963 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-24 12:16:30,964 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:30,964 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:30,964 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-24 12:16:30,964 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:30,965 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,965 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,965 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,965 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,965 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:30,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,965 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:30,965 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:30,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:30,965 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:30,966 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,965 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:30,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:30,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:30,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:30,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:30,966 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:30,966 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:30,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:30,966 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:30,967 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:30,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:30,967 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:30,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:30,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:30,968 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,968 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:30,969 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:30,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:30,969 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:30,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:30,969 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:30,971 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:30,971 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,971 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:30,971 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:30,971 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,971 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:30,971 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:30,971 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:30,971 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:30,971 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:30,971 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-24 12:16:30,972 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:30,971 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:30,971 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:30,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:30,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:30,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:30,972 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:30,972 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:30,971 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:30,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:30,972 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,971 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:30,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:30,971 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:30,972 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:30,972 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:30,972 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:30,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:30,972 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:30,972 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:30,972 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,072 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:31,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:31,084 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:31,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:31,085 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-24 12:16:31,085 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-24 12:16:31,085 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,085 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-24 12:16:31,085 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 12:16:31,086 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,086 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,087 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,087 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,087 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,087 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,087 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,087 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,087 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,088 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:31,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,088 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,089 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,089 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,089 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,089 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,089 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,089 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,089 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,089 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,089 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,089 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,089 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,089 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,089 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,089 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,089 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,089 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,089 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,089 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,089 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,090 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:31,090 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,089 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:31,090 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:31,090 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. started... 2023-07-24 12:16:31,090 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d8d7c0b2b9c0adab4dd247a24a5b7141 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:31,092 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,092 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,092 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,092 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,093 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,092 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,098 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf1/3d7eb053d90b448286b6e3ef5ea34c94 2023-07-24 12:16:31,113 ERROR [BP-51740533-172.31.2.20-1690200889613 heartbeating to localhost/127.0.0.1:33821] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-51740533-172.31.2.20-1690200889613 (Datanode Uuid 295fb89a-ce04-4534-9158-8d1eb7f1a7e1) service to localhost/127.0.0.1:33821 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-24 12:16:31,118 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf2/483d21f9d9e4477695a683a881b66ce6 2023-07-24 12:16:31,122 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf1/3d7eb053d90b448286b6e3ef5ea34c94 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/3d7eb053d90b448286b6e3ef5ea34c94 2023-07-24 12:16:31,153 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/3d7eb053d90b448286b6e3ef5ea34c94, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:16:31,154 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf2/483d21f9d9e4477695a683a881b66ce6 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/483d21f9d9e4477695a683a881b66ce6 2023-07-24 12:16:31,158 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/483d21f9d9e4477695a683a881b66ce6, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:16:31,159 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d8d7c0b2b9c0adab4dd247a24a5b7141 in 68ms, sequenceid=19, compaction requested=false 2023-07-24 12:16:31,159 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d8d7c0b2b9c0adab4dd247a24a5b7141: 2023-07-24 12:16:31,159 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:31,159 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:31,159 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:31,159 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,159 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,159 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,163 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,163 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,163 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,164 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,164 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,164 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,165 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,166 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,166 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-24 12:16:31,166 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1257cb09[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:31,166 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 12:16:31,166 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,167 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,167 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,167 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,167 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,167 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,168 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,167 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,168 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:31,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,168 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:31,168 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,168 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,168 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:31,167 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,168 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,168 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,168 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:31,170 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,170 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,170 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,170 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,170 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,170 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,170 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,170 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,170 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,171 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,171 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,171 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,172 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,173 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:31,174 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 12:16:31,174 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-24 12:16:31,174 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:31,174 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:31,174 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,174 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:31,175 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,175 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,175 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,175 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,175 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,175 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,175 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,175 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,175 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,176 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,176 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,176 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,176 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,177 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,177 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,178 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,178 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,178 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,179 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,179 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,181 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,181 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,181 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,181 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,181 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,181 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-24 12:16:31,182 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:31,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,182 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,182 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,182 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,182 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:31,182 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:31,181 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,182 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,182 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,182 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,182 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,182 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,182 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,182 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,183 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,183 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,183 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,183 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,183 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,282 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:31,283 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:31,293 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:31,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:31,295 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-24 12:16:31,295 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-24 12:16:31,295 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,295 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-24 12:16:31,295 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 12:16:31,295 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,295 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,298 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,298 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,298 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,298 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,298 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,298 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:31,298 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,298 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,299 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,299 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,299 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,299 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,299 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,299 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,300 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,300 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,300 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,300 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,300 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,300 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,300 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,300 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,300 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,300 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:31,300 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,300 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:31,300 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:31,300 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. started... 2023-07-24 12:16:31,300 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,300 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,300 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,300 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,300 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,300 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d8d7c0b2b9c0adab4dd247a24a5b7141 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:31,300 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,300 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,301 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,301 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,301 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,301 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,302 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,302 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,302 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,302 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,302 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,302 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,309 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf1/dc777e780436495e8c0ea1e36ffa697b 2023-07-24 12:16:31,319 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf2/731e2d7b883a49f4bda83afb824a68a1 2023-07-24 12:16:31,322 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf1/dc777e780436495e8c0ea1e36ffa697b as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/dc777e780436495e8c0ea1e36ffa697b 2023-07-24 12:16:31,326 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/dc777e780436495e8c0ea1e36ffa697b, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:16:31,326 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf2/731e2d7b883a49f4bda83afb824a68a1 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/731e2d7b883a49f4bda83afb824a68a1 2023-07-24 12:16:31,329 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/731e2d7b883a49f4bda83afb824a68a1, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:16:31,330 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d8d7c0b2b9c0adab4dd247a24a5b7141 in 30ms, sequenceid=32, compaction requested=true 2023-07-24 12:16:31,330 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d8d7c0b2b9c0adab4dd247a24a5b7141: 2023-07-24 12:16:31,330 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:31,330 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:31,330 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:31,330 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,330 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,330 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,332 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,332 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,332 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,332 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,332 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,333 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-24 12:16:31,334 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6a52484f[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:31,334 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 12:16:31,334 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,335 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,335 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,335 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,335 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,335 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,335 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,335 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,335 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:31,335 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,335 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:31,335 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,335 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,335 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:31,335 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,335 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,335 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:31,338 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,338 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,338 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,339 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,339 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,339 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,339 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,339 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,339 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,339 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,339 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,340 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,341 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:31,342 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 12:16:31,342 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-24 12:16:31,342 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:31,342 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:31,342 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,342 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:31,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,343 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,344 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,343 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,343 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,344 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,344 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,344 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,344 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,345 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,346 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,346 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,346 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,347 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,347 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,353 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,353 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,353 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,353 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,353 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,353 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,353 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,353 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-24 12:16:31,354 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:31,353 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,353 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,353 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,353 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,353 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,353 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,354 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:31,354 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,354 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,354 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,354 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:31,354 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,355 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,354 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,354 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,355 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,355 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,355 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,455 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:31,456 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:31,466 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:31,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:31,468 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-24 12:16:31,468 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-24 12:16:31,468 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,468 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-24 12:16:31,468 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 12:16:31,468 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,468 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,470 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,470 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,470 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,470 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,470 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,470 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:31,470 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,470 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,471 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,471 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,471 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,471 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,471 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,472 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,472 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,472 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,472 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,472 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,472 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,472 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,472 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,472 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,472 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,472 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,472 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,472 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,472 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,472 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,472 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,473 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,473 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:31,473 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:31,473 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:31,473 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. started... 2023-07-24 12:16:31,473 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d8d7c0b2b9c0adab4dd247a24a5b7141 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:31,474 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,474 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,474 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,474 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,475 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,475 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,483 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf1/ba5378e268374a1999f3ea6a457a558d 2023-07-24 12:16:31,494 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91 2023-07-24 12:16:31,498 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf1/ba5378e268374a1999f3ea6a457a558d as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/ba5378e268374a1999f3ea6a457a558d 2023-07-24 12:16:31,501 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/ba5378e268374a1999f3ea6a457a558d, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:16:31,502 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91 2023-07-24 12:16:31,505 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:16:31,506 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d8d7c0b2b9c0adab4dd247a24a5b7141 in 33ms, sequenceid=45, compaction requested=true 2023-07-24 12:16:31,506 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d8d7c0b2b9c0adab4dd247a24a5b7141: 2023-07-24 12:16:31,506 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:31,506 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:31,506 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:31,506 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,506 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,506 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,509 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,509 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,509 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,509 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,509 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,510 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,511 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,511 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-24 12:16:31,511 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7cbc7bbf[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:31,511 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 12:16:31,511 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,512 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,512 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,512 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,512 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,512 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:31,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,512 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,513 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,513 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,513 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,513 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,513 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:31,513 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:31,512 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,513 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,513 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,513 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,513 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:31,514 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,514 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,514 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,514 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,514 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,514 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,514 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,514 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,514 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,514 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,514 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,515 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,516 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,517 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:31,517 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 12:16:31,517 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-24 12:16:31,517 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:31,517 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:31,517 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,517 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:31,519 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,519 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,519 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,519 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,519 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,519 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,519 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,519 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,519 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,519 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,520 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,520 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,520 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,520 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,521 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,521 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,522 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,522 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,526 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,526 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-24 12:16:31,527 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:31,526 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,527 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,526 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,526 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,527 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:31,527 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:31,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,527 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,527 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,527 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,527 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,527 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,528 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,528 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,627 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:31,628 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:31,639 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:31,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:31,640 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-24 12:16:31,641 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-24 12:16:31,641 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,641 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-24 12:16:31,641 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 12:16:31,641 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,641 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,643 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,643 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,643 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,643 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,643 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,643 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,643 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:31,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,643 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,643 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,644 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,644 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,644 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,644 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,644 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,644 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,644 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,644 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,644 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,644 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,644 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,644 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,644 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,644 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,645 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,644 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,645 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,645 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,645 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,645 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,645 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,645 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,645 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,645 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,645 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,645 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:31,645 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:31,645 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:31,645 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. started... 2023-07-24 12:16:31,645 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d8d7c0b2b9c0adab4dd247a24a5b7141 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:31,646 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,647 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,647 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,647 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,647 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,647 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,654 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf1/4f9ffc93bda548d280a3256a5e7c3cc2 2023-07-24 12:16:31,666 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf2/0a871c82146d4fcda0b36e2e944ee8b4 2023-07-24 12:16:31,670 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf1/4f9ffc93bda548d280a3256a5e7c3cc2 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/4f9ffc93bda548d280a3256a5e7c3cc2 2023-07-24 12:16:31,674 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/4f9ffc93bda548d280a3256a5e7c3cc2, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:16:31,674 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf2/0a871c82146d4fcda0b36e2e944ee8b4 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/0a871c82146d4fcda0b36e2e944ee8b4 2023-07-24 12:16:31,678 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/0a871c82146d4fcda0b36e2e944ee8b4, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:16:31,678 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d8d7c0b2b9c0adab4dd247a24a5b7141 in 33ms, sequenceid=58, compaction requested=true 2023-07-24 12:16:31,679 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d8d7c0b2b9c0adab4dd247a24a5b7141: 2023-07-24 12:16:31,679 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:31,679 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:31,679 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:31,679 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,679 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,679 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,684 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,684 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,684 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,685 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,685 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,685 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,686 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-24 12:16:31,686 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6487f3a4[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:31,686 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 12:16:31,687 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,688 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,688 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,688 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,688 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,688 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,688 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,688 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,688 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:31,688 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,688 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,688 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,688 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,688 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:31,688 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,688 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:31,688 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:31,689 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,689 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,689 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,690 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,690 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,690 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,690 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,690 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,690 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,690 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,690 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,691 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,692 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,693 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:31,693 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 12:16:31,693 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-24 12:16:31,693 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:31,693 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:31,693 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,693 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:31,696 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,696 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,696 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,696 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,696 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,696 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,696 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,696 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,696 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,696 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,697 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,697 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,697 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,697 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,698 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,699 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,699 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,699 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,699 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,701 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,701 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,701 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,701 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,701 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,702 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,702 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,702 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-24 12:16:31,702 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:31,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,701 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,702 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,702 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,702 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,702 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,702 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,702 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,702 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,702 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:31,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,702 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,702 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,702 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:31,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,803 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:31,804 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:31,815 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:31,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testRecoveryAndDoubleExecution was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:31,816 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testRecoveryAndDoubleExecution 2023-07-24 12:16:31,816 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testRecoveryAndDoubleExecution' 2023-07-24 12:16:31,816 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,816 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testRecoveryAndDoubleExecution' starting 'acquire' 2023-07-24 12:16:31,816 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testRecoveryAndDoubleExecution', kicking off acquire phase on members. 2023-07-24 12:16:31,817 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,817 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,818 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,818 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,818 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,818 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,818 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,818 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,818 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,819 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:31,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,819 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,819 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:31,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,819 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,819 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,819 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,819 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testRecoveryAndDoubleExecution 2023-07-24 12:16:31,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,820 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,820 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testRecoveryAndDoubleExecution is still present, removing 2023-07-24 12:16:31,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,820 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testRecoveryAndDoubleExecution' with timeout 60000ms 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testRecoveryAndDoubleExecution' starting 'acquire' stage 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,820 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:31,820 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:31,820 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. started... 2023-07-24 12:16:31,821 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing d8d7c0b2b9c0adab4dd247a24a5b7141 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:31,821 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,823 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,823 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,824 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,824 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,824 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,830 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf1/a663a0b330824bd8a6895bb1de6c87fa 2023-07-24 12:16:31,844 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf2/1f00641dc7a04e22a71975cc7596096c 2023-07-24 12:16:31,848 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf1/a663a0b330824bd8a6895bb1de6c87fa as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a663a0b330824bd8a6895bb1de6c87fa 2023-07-24 12:16:31,852 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a663a0b330824bd8a6895bb1de6c87fa, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:16:31,853 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf2/1f00641dc7a04e22a71975cc7596096c as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/1f00641dc7a04e22a71975cc7596096c 2023-07-24 12:16:31,856 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/1f00641dc7a04e22a71975cc7596096c, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:16:31,857 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for d8d7c0b2b9c0adab4dd247a24a5b7141 in 37ms, sequenceid=71, compaction requested=true 2023-07-24 12:16:31,857 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for d8d7c0b2b9c0adab4dd247a24a5b7141: 2023-07-24 12:16:31,857 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:31,857 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:31,857 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:31,857 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,857 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testRecoveryAndDoubleExecution' locally acquired 2023-07-24 12:16:31,857 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testRecoveryAndDoubleExecution) in zk 2023-07-24 12:16:31,858 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,858 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,858 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,859 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,859 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testRecoveryAndDoubleExecution' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:31,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,859 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,860 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testRecoveryAndDoubleExecution' on coordinator 2023-07-24 12:16:31,861 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48a473be[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:31,861 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testRecoveryAndDoubleExecution' starting 'in-barrier' execution. 2023-07-24 12:16:31,861 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,862 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,862 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,862 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,862 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,862 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,862 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,862 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,862 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,862 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,862 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:31,862 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,862 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:31,862 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testRecoveryAndDoubleExecution' received 'reached' from coordinator. 2023-07-24 12:16:31,862 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testRecoveryAndDoubleExecution' locally completed 2023-07-24 12:16:31,862 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:31,862 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testRecoveryAndDoubleExecution' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:31,864 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,864 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,864 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,864 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,864 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,864 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,864 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,864 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,866 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testRecoveryAndDoubleExecution' has notified controller of completion 2023-07-24 12:16:31,866 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,866 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testRecoveryAndDoubleExecution' completed. 2023-07-24 12:16:31,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testRecoveryAndDoubleExecution' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:31,868 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testRecoveryAndDoubleExecution', counting down latch. Waiting for 0 more 2023-07-24 12:16:31,868 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testRecoveryAndDoubleExecution' execution completed 2023-07-24 12:16:31,869 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:31,869 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:31,869 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testRecoveryAndDoubleExecution 2023-07-24 12:16:31,869 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testRecoveryAndDoubleExecutionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:31,870 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,870 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:31,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:31,870 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,870 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,870 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,870 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:31,870 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,870 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,870 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,871 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,871 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,871 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,871 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:31,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,872 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,872 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:31,873 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,873 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testRecoveryAndDoubleExecution 2023-07-24 12:16:31,874 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,874 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,874 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,878 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,878 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,878 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,878 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,878 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,878 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:31,878 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,878 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,879 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:31,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,879 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,879 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:31,879 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,879 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:31,879 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,879 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:31,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testRecoveryAndDoubleExecution' 2023-07-24 12:16:31,879 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:31,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:31,879 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:31,879 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:31,879 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,880 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testRecoveryAndDoubleExecution'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:31,880 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:31,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,880 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testRecoveryAndDoubleExecution 2023-07-24 12:16:31,880 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:31,880 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:31,980 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:31,981 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:32,239 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: true 2023-07-24 12:16:32,240 DEBUG [Listener at localhost/37279] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:32,241 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,242 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,242 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,242 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,243 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:16:32,243 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,243 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:16:32,243 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:16:32,491 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(454): Restart 0 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,491 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:16:32,492 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:16:32,492 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:16:32,492 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-9,5,PEWorkerGroup] Thread[HFileArchiver-10,5,PEWorkerGroup] Thread[HFileArchiver-11,5,PEWorkerGroup] Thread[HFileArchiver-12,5,PEWorkerGroup] 2023-07-24 12:16:32,492 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:16:32,492 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:16:32,492 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:16:32,492 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:16:32,493 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:16:32,495 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[] 2023-07-24 12:16:32,495 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:16:32,496 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:16:32,496 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:16:32,496 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:16:32,497 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:32,497 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:16:32,497 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:32,497 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:32,498 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:32,498 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:16:32,498 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:32,498 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:32,498 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:32,499 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:16:32,499 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:16:32,499 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:32,499 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:32,500 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:32,500 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:16:32,500 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:32,500 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:32,501 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:32,501 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:16:32,501 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:32,501 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:32,501 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:32,502 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:32,502 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:32,502 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:32,503 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:32,503 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:32,503 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:32,503 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:32,504 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:32,504 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,504 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:16:32,504 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,504 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,504 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,504 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 11 msec 2023-07-24 12:16:32,505 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:16:32,506 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:16:32,506 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690201022506 2023-07-24 12:16:32,507 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,507 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:16:32,510 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:32,510 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:32,510 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:16:32,510 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690201052510 2023-07-24 12:16:32,510 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201112510 2023-07-24 12:16:32,510 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 12:16:32,511 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:16:32,512 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:16:32,512 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,512 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:16:32,512 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:16:32,761 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(454): Restart 1 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,761 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:16:32,761 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:16:32,761 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:16:32,761 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:16:32,761 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:16:32,761 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:16:32,761 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:16:32,761 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:16:32,762 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:16:32,763 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0] 2023-07-24 12:16:32,764 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:16:32,764 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:16:32,764 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:16:32,765 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:16:32,765 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:32,765 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:16:32,765 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:32,765 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:32,765 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:32,766 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:16:32,766 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:32,766 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:32,766 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:32,766 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:16:32,766 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:16:32,767 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:32,767 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:32,767 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:32,767 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:16:32,767 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:32,767 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:32,767 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:32,768 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:16:32,768 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:32,768 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:32,768 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:32,768 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:32,768 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:32,768 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:32,769 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:32,769 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:32,769 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:32,769 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:32,770 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:32,770 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,770 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:16:32,770 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,770 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,770 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,771 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 8 msec 2023-07-24 12:16:32,771 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:16:32,772 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:16:32,774 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690201022774 2023-07-24 12:16:32,775 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,775 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:16:32,778 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:32,778 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:32,778 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:16:32,778 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690201052778 2023-07-24 12:16:32,778 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201112778 2023-07-24 12:16:32,778 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 12:16:32,779 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:16:32,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN}] 2023-07-24 12:16:32,781 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:16:32,781 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:32,781 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:16:32,781 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:16:33,029 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(454): Restart 2 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:33,029 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:16:33,029 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:16:33,029 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:16:33,029 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:16:33,029 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:16:33,029 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:16:33,029 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:16:33,029 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:16:33,030 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:16:33,031 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1] 2023-07-24 12:16:33,031 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:16:33,032 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:16:33,032 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:16:33,032 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:16:33,032 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:33,032 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:16:33,033 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:33,033 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:33,033 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:33,033 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:16:33,033 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:33,033 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:33,033 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:33,034 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:16:33,034 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:16:33,034 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:33,034 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:33,034 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:33,034 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:16:33,034 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:33,034 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:33,034 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:33,035 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:16:33,035 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:33,035 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:33,035 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:33,035 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:33,035 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:33,035 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:33,036 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:33,036 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:33,036 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:33,036 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:33,036 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:33,036 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:33,036 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:16:33,036 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:33,036 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:33,037 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:33,037 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-24 12:16:33,037 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:16:33,038 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:16:33,038 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690201023038 2023-07-24 12:16:33,039 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CLOSE_PARENT_REGION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:33,039 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:16:33,041 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:33,042 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:33,042 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:16:33,042 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690201053042 2023-07-24 12:16:33,042 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201113042 2023-07-24 12:16:33,042 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 12:16:33,042 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN}] 2023-07-24 12:16:33,042 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:16:33,044 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=113, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN 2023-07-24 12:16:33,045 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:33,045 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200993045"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200993045"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200993045"}]},"ts":"1690200993045"} 2023-07-24 12:16:33,046 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=114, ppid=113, state=RUNNABLE; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:33,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing d8d7c0b2b9c0adab4dd247a24a5b7141, disabling compactions & flushes 2023-07-24 12:16:33,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:33,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:33,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. after waiting 0 ms 2023-07-24 12:16:33,198 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:33,198 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2745): Flushing d8d7c0b2b9c0adab4dd247a24a5b7141 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-24 12:16:33,206 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf1/a18e29f739c94ff5a1d1eb3a03413310 2023-07-24 12:16:33,217 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf2/716242cab60c4c8283be7787fbfac46d 2023-07-24 12:16:33,221 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf1/a18e29f739c94ff5a1d1eb3a03413310 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a18e29f739c94ff5a1d1eb3a03413310 2023-07-24 12:16:33,224 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a18e29f739c94ff5a1d1eb3a03413310, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 12:16:33,224 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/.tmp/cf2/716242cab60c4c8283be7787fbfac46d as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/716242cab60c4c8283be7787fbfac46d 2023-07-24 12:16:33,228 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/716242cab60c4c8283be7787fbfac46d, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 12:16:33,228 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for d8d7c0b2b9c0adab4dd247a24a5b7141 in 30ms, sequenceid=82, compaction requested=true 2023-07-24 12:16:33,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-24 12:16:33,232 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:16:33,233 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. 2023-07-24 12:16:33,233 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for d8d7c0b2b9c0adab4dd247a24a5b7141: 2023-07-24 12:16:33,235 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,235 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=113 updating hbase:meta row=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=CLOSED 2023-07-24 12:16:33,235 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200993235"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200993235"}]},"ts":"1690200993235"} 2023-07-24 12:16:33,237 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=114, resume processing ppid=113 2023-07-24 12:16:33,238 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 in 190 msec 2023-07-24 12:16:33,239 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=113, resume processing ppid=112 2023-07-24 12:16:33,239 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN in 196 msec 2023-07-24 12:16:33,241 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:16:33,241 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:33,241 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:16:33,241 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:16:33,292 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(454): Restart 3 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:33,292 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:16:33,292 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:16:33,292 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:16:33,293 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:16:33,293 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:16:33,293 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:16:33,293 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:16:33,293 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:16:33,293 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:16:33,295 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2] 2023-07-24 12:16:33,295 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:16:33,295 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:16:33,295 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:16:33,296 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:16:33,296 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:16:33,296 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:16:33,296 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:33,296 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:16:33,297 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:33,297 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:33,297 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:33,297 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:16:33,297 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:33,298 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:33,298 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:33,298 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:16:33,298 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:16:33,299 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:33,299 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:33,299 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:33,299 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:16:33,300 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:33,300 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:33,300 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:33,300 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:16:33,300 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:33,301 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:33,301 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:33,301 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:33,302 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:33,302 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:33,302 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:33,302 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:33,302 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:33,303 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:33,303 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:33,303 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:33,303 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN 2023-07-24 12:16:33,303 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:33,303 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:16:33,303 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:33,303 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:33,303 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:33,304 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:33,304 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:33,304 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-24 12:16:33,304 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:16:33,305 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:16:33,305 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690201023305 2023-07-24 12:16:33,305 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:16:33,307 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGIONS_CHECK_CLOSED_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:33,309 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:33,309 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:33,309 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:16:33,310 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690201053310 2023-07-24 12:16:33,310 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201113310 2023-07-24 12:16:33,310 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 12:16:33,312 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:16:33,320 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=d8d7c0b2b9c0adab4dd247a24a5b7141, threads=14 2023-07-24 12:16:33,321 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/0a871c82146d4fcda0b36e2e944ee8b4 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,321 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/1f00641dc7a04e22a71975cc7596096c for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,321 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/483d21f9d9e4477695a683a881b66ce6 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,321 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/716242cab60c4c8283be7787fbfac46d for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,321 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/731e2d7b883a49f4bda83afb824a68a1 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,321 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/99f25d22508e4e5c90b523abeff3417b for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,321 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,321 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/3d7eb053d90b448286b6e3ef5ea34c94 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,321 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/4f9ffc93bda548d280a3256a5e7c3cc2 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,321 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,321 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a18e29f739c94ff5a1d1eb3a03413310 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,322 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a663a0b330824bd8a6895bb1de6c87fa for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,322 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/ba5378e268374a1999f3ea6a457a558d for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,322 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/dc777e780436495e8c0ea1e36ffa697b for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,491 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,495 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,495 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/3d7eb053d90b448286b6e3ef5ea34c94 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,511 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/4f9ffc93bda548d280a3256a5e7c3cc2 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,527 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a18e29f739c94ff5a1d1eb3a03413310 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,531 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/483d21f9d9e4477695a683a881b66ce6 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,541 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/0a871c82146d4fcda0b36e2e944ee8b4 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,543 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/ba5378e268374a1999f3ea6a457a558d for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,543 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/99f25d22508e4e5c90b523abeff3417b for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,546 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/716242cab60c4c8283be7787fbfac46d for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,600 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 12:16:33,875 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/dc777e780436495e8c0ea1e36ffa697b for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,876 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a663a0b330824bd8a6895bb1de6c87fa for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,876 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/731e2d7b883a49f4bda83afb824a68a1 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,878 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/1f00641dc7a04e22a71975cc7596096c for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:33,878 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region d8d7c0b2b9c0adab4dd247a24a5b7141 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 12:16:33,906 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:16:33,906 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:33,906 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:16:33,906 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:16:34,060 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(454): Restart 4 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:34,060 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:16:34,060 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:16:34,060 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:16:34,061 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:16:34,061 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:16:34,061 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:16:34,061 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:16:34,061 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:16:34,062 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:16:34,063 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7] 2023-07-24 12:16:34,063 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:16:34,063 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:16:34,063 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:16:34,063 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:16:34,064 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:16:34,064 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:16:34,064 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:34,064 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:16:34,064 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:34,064 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:34,064 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:34,064 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:16:34,065 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:34,065 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:34,065 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:34,065 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:16:34,065 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:16:34,065 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:34,065 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:34,065 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:34,066 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:16:34,066 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:34,066 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:34,066 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:34,066 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:16:34,066 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:34,066 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:34,066 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:34,067 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:34,067 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:34,067 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:34,067 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:34,067 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:34,067 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:34,067 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:34,067 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:34,068 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:34,068 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN 2023-07-24 12:16:34,068 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:34,068 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:16:34,068 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:34,068 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:34,068 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:34,068 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:34,068 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:34,068 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-24 12:16:34,069 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:16:34,069 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:16:34,070 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690201024070 2023-07-24 12:16:34,071 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:16:34,071 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_CREATE_DAUGHTER_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:34,073 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:34,073 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:34,073 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:16:34,073 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690201054073 2023-07-24 12:16:34,073 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201114073 2023-07-24 12:16:34,074 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 12:16:34,081 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=112 splitting 14 storefiles, region=d8d7c0b2b9c0adab4dd247a24a5b7141, threads=14 2023-07-24 12:16:34,082 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/0a871c82146d4fcda0b36e2e944ee8b4 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,082 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/1f00641dc7a04e22a71975cc7596096c for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,082 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/483d21f9d9e4477695a683a881b66ce6 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,082 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/716242cab60c4c8283be7787fbfac46d for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,083 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/731e2d7b883a49f4bda83afb824a68a1 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,083 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/99f25d22508e4e5c90b523abeff3417b for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,084 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,089 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/3d7eb053d90b448286b6e3ef5ea34c94 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,089 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/4f9ffc93bda548d280a3256a5e7c3cc2 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,089 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,089 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a18e29f739c94ff5a1d1eb3a03413310 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,090 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a663a0b330824bd8a6895bb1de6c87fa for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,091 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/ba5378e268374a1999f3ea6a457a558d for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,091 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=112 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/dc777e780436495e8c0ea1e36ffa697b for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,109 ERROR [BP-51740533-172.31.2.20-1690200889613 heartbeating to localhost/127.0.0.1:33821] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-51740533-172.31.2.20-1690200889613 (Datanode Uuid dce5ccf6-5af1-4cab-b72f-523ad9084719) service to localhost/127.0.0.1:33821 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-24 12:16:34,224 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/99f25d22508e4e5c90b523abeff3417b for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,231 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/1f00641dc7a04e22a71975cc7596096c for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,233 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/483d21f9d9e4477695a683a881b66ce6 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,246 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,246 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/4f9ffc93bda548d280a3256a5e7c3cc2 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,259 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/731e2d7b883a49f4bda83afb824a68a1 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,264 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/ba5378e268374a1999f3ea6a457a558d for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,265 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,266 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a18e29f739c94ff5a1d1eb3a03413310 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,266 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/0a871c82146d4fcda0b36e2e944ee8b4 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,267 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/3d7eb053d90b448286b6e3ef5ea34c94 for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,268 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a663a0b330824bd8a6895bb1de6c87fa for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,555 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/dc777e780436495e8c0ea1e36ffa697b for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,585 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=112 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/716242cab60c4c8283be7787fbfac46d for region: d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:34,585 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=112 split storefiles for region d8d7c0b2b9c0adab4dd247a24a5b7141 Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 12:16:35,021 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:16:35,026 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 12:16:35,027 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 12:16:35,027 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:16:35,027 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,027 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:16:35,028 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:16:35,074 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(454): Restart 5 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,074 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:16:35,074 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:16:35,074 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:16:35,074 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:16:35,074 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:16:35,075 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:16:35,075 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:16:35,075 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:16:35,075 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:16:35,076 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8] 2023-07-24 12:16:35,076 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:16:35,077 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:16:35,077 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:16:35,077 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:16:35,077 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:16:35,077 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:16:35,077 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:35,078 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:16:35,078 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:35,078 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:35,078 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:35,078 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:16:35,078 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:35,078 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:35,078 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:35,079 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:16:35,079 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:16:35,079 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:35,079 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:35,079 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:35,079 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:16:35,079 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:35,079 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:35,080 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:35,080 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:16:35,080 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:35,080 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:35,080 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,080 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,080 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:35,080 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,081 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,081 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:35,081 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:35,081 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:35,081 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:35,081 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,081 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN 2023-07-24 12:16:35,081 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:35,081 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:16:35,082 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,082 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,082 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,082 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:35,082 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:35,082 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-24 12:16:35,083 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:16:35,083 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:16:35,083 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690201025083 2023-07-24 12:16:35,084 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:16:35,085 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_WRITE_MAX_SEQUENCE_ID_FILE; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,087 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:35,088 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:35,088 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:16:35,088 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690201055088 2023-07-24 12:16:35,088 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201115088 2023-07-24 12:16:35,088 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 12:16:35,090 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:16:35,092 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:16:35,092 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,092 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:16:35,092 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:16:35,338 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(454): Restart 6 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,338 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:16:35,338 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:16:35,338 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:16:35,338 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:16:35,339 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:16:35,339 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:16:35,339 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:16:35,339 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:16:35,339 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:16:35,340 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9] 2023-07-24 12:16:35,340 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:16:35,341 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:16:35,341 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:16:35,341 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:16:35,341 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:16:35,341 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:16:35,341 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:35,341 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:16:35,342 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:35,342 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:35,342 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:35,342 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:16:35,342 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:35,342 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:35,342 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:35,343 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:16:35,343 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:16:35,343 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:35,343 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:35,343 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:35,343 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:16:35,343 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:35,343 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:35,343 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:35,344 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:16:35,344 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:35,344 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:35,344 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,344 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,344 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:35,344 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,344 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,345 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:35,345 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:35,345 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:35,345 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:35,345 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,345 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN 2023-07-24 12:16:35,345 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:35,345 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:16:35,346 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,346 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,346 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,346 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:35,346 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:35,346 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-24 12:16:35,346 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:16:35,347 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:16:35,348 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690201025348 2023-07-24 12:16:35,349 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_BEFORE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,349 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:16:35,352 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:35,352 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:35,352 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:16:35,352 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690201055352 2023-07-24 12:16:35,352 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201115352 2023-07-24 12:16:35,352 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 2 msec 2023-07-24 12:16:35,353 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:16:35,354 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200995354"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1690200995354"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1690200995354"}]},"ts":"1690200995354"} 2023-07-24 12:16:35,355 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690200995354"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200995354"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200995354"}]},"ts":"1690200995354"} 2023-07-24 12:16:35,355 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690200995354"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200995354"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200995354"}]},"ts":"1690200995354"} 2023-07-24 12:16:35,356 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:16:35,356 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,356 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:16:35,356 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:16:35,603 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(454): Restart 7 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,603 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:16:35,603 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:16:35,603 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:16:35,603 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:16:35,603 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:16:35,604 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:16:35,604 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:16:35,604 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:16:35,604 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:16:35,606 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10] 2023-07-24 12:16:35,606 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:16:35,606 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:16:35,607 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:16:35,607 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:16:35,608 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:16:35,608 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:16:35,608 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:35,608 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:16:35,609 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:35,609 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:35,609 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:35,609 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:16:35,609 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:35,609 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:35,609 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:35,610 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:16:35,610 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:16:35,610 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:35,610 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:35,610 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:35,611 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:16:35,611 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:35,611 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:35,611 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:35,611 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:16:35,612 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:35,612 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:35,612 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,613 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,613 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:35,613 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,613 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,613 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:35,614 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:35,614 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:35,614 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:35,614 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,614 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN 2023-07-24 12:16:35,615 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:35,615 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:16:35,615 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,615 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,615 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,615 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:35,615 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:35,615 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-24 12:16:35,616 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:16:35,617 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:16:35,619 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690201025619 2023-07-24 12:16:35,621 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_UPDATE_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,621 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:16:35,624 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:35,624 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:35,624 INFO [Listener at localhost/37279] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:35,624 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=6b1c90f5fec687b8c357b1ca17c402ea, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-24 12:16:35,625 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=21bfd26c5d6e7afe0247ea257443adaf, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-24 12:16:35,625 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:16:35,625 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690201055625 2023-07-24 12:16:35,625 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201115625 2023-07-24 12:16:35,625 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 12:16:35,625 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141.","families":{"info":[{"qualifier":"regioninfo","vlen":64,"tag":[],"timestamp":"1690200995625"},{"qualifier":"splitA","vlen":66,"tag":[],"timestamp":"1690200995625"},{"qualifier":"splitB","vlen":66,"tag":[],"timestamp":"1690200995625"}]},"ts":"1690200995625"} 2023-07-24 12:16:35,626 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690200995625"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200995625"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200995625"}]},"ts":"1690200995625"} 2023-07-24 12:16:35,626 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690200995625"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200995625"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200995625"}]},"ts":"1690200995625"} 2023-07-24 12:16:35,627 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:16:35,629 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:16:35,629 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,629 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:16:35,629 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:16:35,875 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(454): Restart 8 exec state=pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,875 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:16:35,875 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:16:35,876 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:16:35,876 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:16:35,876 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:16:35,876 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:16:35,876 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:16:35,876 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:16:35,877 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:16:35,878 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11] 2023-07-24 12:16:35,878 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:16:35,878 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:16:35,878 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:16:35,878 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:16:35,879 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:16:35,879 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:16:35,879 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:35,879 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:16:35,879 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:35,879 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:35,879 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:35,880 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:16:35,880 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:35,880 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:35,880 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:35,880 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:16:35,880 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:16:35,880 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:35,880 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:35,881 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:35,881 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:16:35,881 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:35,881 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:35,881 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:35,881 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:16:35,881 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:35,881 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:35,881 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,882 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,882 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:35,882 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,882 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:35,882 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:35,882 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:35,882 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:35,883 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:35,883 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,883 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN 2023-07-24 12:16:35,883 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:35,883 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:16:35,883 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,883 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,883 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,883 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:35,883 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:35,883 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 6 msec 2023-07-24 12:16:35,884 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:16:35,884 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:16:35,885 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690201025885 2023-07-24 12:16:35,886 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:16:35,886 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_PRE_OPERATION_AFTER_META; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,888 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:35,889 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:35,889 INFO [Listener at localhost/37279] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:35,889 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=6b1c90f5fec687b8c357b1ca17c402ea, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-24 12:16:35,889 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=21bfd26c5d6e7afe0247ea257443adaf, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-24 12:16:35,889 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:16:35,889 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690201055889 2023-07-24 12:16:35,889 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201115889 2023-07-24 12:16:35,889 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 3 msec 2023-07-24 12:16:35,889 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:16:35,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6b1c90f5fec687b8c357b1ca17c402ea, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=21bfd26c5d6e7afe0247ea257443adaf, ASSIGN}] 2023-07-24 12:16:35,891 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:16:35,891 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:35,891 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:16:35,891 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:16:36,139 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(454): Restart 9 exec state=pid=112, state=WAITING:SPLIT_TABLE_REGION_POST_OPERATION, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,140 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:16:36,140 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:16:36,140 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:16:36,140 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:16:36,140 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:16:36,140 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:16:36,140 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:16:36,140 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:16:36,141 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:16:36,142 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12] 2023-07-24 12:16:36,142 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:16:36,142 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:16:36,142 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:16:36,143 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:16:36,143 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:16:36,143 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:16:36,143 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:36,143 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:16:36,144 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:36,144 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:36,144 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:36,144 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:16:36,144 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:36,145 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:36,145 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:36,145 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:16:36,145 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:16:36,145 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:36,145 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:36,145 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:36,146 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:16:36,146 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:36,146 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:36,146 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:36,146 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:16:36,146 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:36,146 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:36,146 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:36,147 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:36,147 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:36,147 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:36,147 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:36,147 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:36,147 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:36,148 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:36,148 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:36,148 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,148 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN 2023-07-24 12:16:36,148 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:36,148 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:16:36,148 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,148 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,148 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,149 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:36,149 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:36,149 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 7 msec 2023-07-24 12:16:36,149 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:16:36,150 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:16:36,150 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690201026150 2023-07-24 12:16:36,151 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:16:36,152 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_OPEN_CHILD_REGIONS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,154 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:36,155 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:36,155 INFO [Listener at localhost/37279] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:36,155 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=6b1c90f5fec687b8c357b1ca17c402ea, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-24 12:16:36,155 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=21bfd26c5d6e7afe0247ea257443adaf, regionState=CLOSED, lastHost=null, regionLocation=null, openSeqNum=1 2023-07-24 12:16:36,155 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:16:36,155 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690201056155 2023-07-24 12:16:36,156 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201116156 2023-07-24 12:16:36,156 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 4 msec 2023-07-24 12:16:36,156 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6b1c90f5fec687b8c357b1ca17c402ea, ASSIGN}, {pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=21bfd26c5d6e7afe0247ea257443adaf, ASSIGN}] 2023-07-24 12:16:36,156 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:16:36,158 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=21bfd26c5d6e7afe0247ea257443adaf, ASSIGN 2023-07-24 12:16:36,158 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=116, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=21bfd26c5d6e7afe0247ea257443adaf, ASSIGN; state=CLOSED, location=jenkins-hbase6.apache.org,34575,1690200894532; forceNewPlan=false, retain=false 2023-07-24 12:16:36,159 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6b1c90f5fec687b8c357b1ca17c402ea, ASSIGN 2023-07-24 12:16:36,159 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=115, ppid=112, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6b1c90f5fec687b8c357b1ca17c402ea, ASSIGN; state=CLOSED, location=jenkins-hbase6.apache.org,34575,1690200894532; forceNewPlan=false, retain=false 2023-07-24 12:16:36,308 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-24 12:16:36,310 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=6b1c90f5fec687b8c357b1ca17c402ea, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:36,310 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690200996310"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200996310"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200996310"}]},"ts":"1690200996310"} 2023-07-24 12:16:36,311 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=117, ppid=115, state=RUNNABLE; OpenRegionProcedure 6b1c90f5fec687b8c357b1ca17c402ea, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:36,313 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=21bfd26c5d6e7afe0247ea257443adaf, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:36,313 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690200996313"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200996313"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200996313"}]},"ts":"1690200996313"} 2023-07-24 12:16:36,314 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=118, ppid=116, state=RUNNABLE; OpenRegionProcedure 21bfd26c5d6e7afe0247ea257443adaf, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:36,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf. 2023-07-24 12:16:36,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 21bfd26c5d6e7afe0247ea257443adaf, NAME => 'testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.', STARTKEY => '41', ENDKEY => ''} 2023-07-24 12:16:36,466 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:16:36,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:16:36,466 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,467 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,468 INFO [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,468 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1 2023-07-24 12:16:36,469 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1 2023-07-24 12:16:36,469 INFO [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 21bfd26c5d6e7afe0247ea257443adaf columnFamilyName cf1 2023-07-24 12:16:36,480 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/3d7eb053d90b448286b6e3ef5ea34c94-top 2023-07-24 12:16:36,483 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/4f9ffc93bda548d280a3256a5e7c3cc2-top 2023-07-24 12:16:36,486 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf-top 2023-07-24 12:16:36,490 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a18e29f739c94ff5a1d1eb3a03413310-top 2023-07-24 12:16:36,493 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a663a0b330824bd8a6895bb1de6c87fa-top 2023-07-24 12:16:36,496 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/ba5378e268374a1999f3ea6a457a558d-top 2023-07-24 12:16:36,499 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/dc777e780436495e8c0ea1e36ffa697b-top 2023-07-24 12:16:36,500 INFO [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(310): Store=21bfd26c5d6e7afe0247ea257443adaf/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:36,500 INFO [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,500 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2 2023-07-24 12:16:36,501 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2 2023-07-24 12:16:36,501 INFO [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 21bfd26c5d6e7afe0247ea257443adaf columnFamilyName cf2 2023-07-24 12:16:36,512 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/0a871c82146d4fcda0b36e2e944ee8b4-top 2023-07-24 12:16:36,516 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/1f00641dc7a04e22a71975cc7596096c-top 2023-07-24 12:16:36,519 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/483d21f9d9e4477695a683a881b66ce6-top 2023-07-24 12:16:36,522 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/716242cab60c4c8283be7787fbfac46d-top 2023-07-24 12:16:36,526 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/731e2d7b883a49f4bda83afb824a68a1-top 2023-07-24 12:16:36,529 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/99f25d22508e4e5c90b523abeff3417b-top 2023-07-24 12:16:36,533 DEBUG [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91-top 2023-07-24 12:16:36,533 INFO [StoreOpener-21bfd26c5d6e7afe0247ea257443adaf-1] regionserver.HStore(310): Store=21bfd26c5d6e7afe0247ea257443adaf/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:36,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,536 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:16:36,537 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 21bfd26c5d6e7afe0247ea257443adaf; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66514642, jitterRate=-0.008854597806930542}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:16:36,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 21bfd26c5d6e7afe0247ea257443adaf: 2023-07-24 12:16:36,538 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf., pid=118, masterSystemTime=1690200996463 2023-07-24 12:16:36,538 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:36,541 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:16:36,541 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:16:36,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:36,542 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf. 2023-07-24 12:16:36,542 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf. 2023-07-24 12:16:36,542 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1912): 21bfd26c5d6e7afe0247ea257443adaf/cf2 is initiating minor compaction (all files) 2023-07-24 12:16:36,542 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1912): 21bfd26c5d6e7afe0247ea257443adaf/cf1 is initiating minor compaction (all files) 2023-07-24 12:16:36,542 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 21bfd26c5d6e7afe0247ea257443adaf/cf2 in testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf. 2023-07-24 12:16:36,542 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 21bfd26c5d6e7afe0247ea257443adaf/cf1 in testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf. 2023-07-24 12:16:36,542 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/99f25d22508e4e5c90b523abeff3417b-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/483d21f9d9e4477695a683a881b66ce6-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/731e2d7b883a49f4bda83afb824a68a1-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/0a871c82146d4fcda0b36e2e944ee8b4-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/1f00641dc7a04e22a71975cc7596096c-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/716242cab60c4c8283be7787fbfac46d-top] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/.tmp, totalSize=34.9 K 2023-07-24 12:16:36,542 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/3d7eb053d90b448286b6e3ef5ea34c94-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/dc777e780436495e8c0ea1e36ffa697b-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/ba5378e268374a1999f3ea6a457a558d-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/4f9ffc93bda548d280a3256a5e7c3cc2-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a663a0b330824bd8a6895bb1de6c87fa-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a18e29f739c94ff5a1d1eb3a03413310-top] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/.tmp, totalSize=34.9 K 2023-07-24 12:16:36,543 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690200990910 2023-07-24 12:16:36,543 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting 9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690200990910 2023-07-24 12:16:36,543 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf. 2023-07-24 12:16:36,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf. 2023-07-24 12:16:36,544 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690200991073 2023-07-24 12:16:36,544 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting 3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690200991073 2023-07-24 12:16:36,544 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=116 updating hbase:meta row=21bfd26c5d6e7afe0247ea257443adaf, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:36,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea. 2023-07-24 12:16:36,544 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690200996544"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200996544"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200996544"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200996544"}]},"ts":"1690200996544"} 2023-07-24 12:16:36,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 6b1c90f5fec687b8c357b1ca17c402ea, NAME => 'testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.', STARTKEY => '', ENDKEY => '41'} 2023-07-24 12:16:36,544 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690200991284 2023-07-24 12:16:36,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:16:36,545 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690200991284 2023-07-24 12:16:36,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testRecoveryAndDoubleExecution 6b1c90f5fec687b8c357b1ca17c402ea 2023-07-24 12:16:36,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:16:36,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 6b1c90f5fec687b8c357b1ca17c402ea 2023-07-24 12:16:36,545 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 6b1c90f5fec687b8c357b1ca17c402ea 2023-07-24 12:16:36,546 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690200991456 2023-07-24 12:16:36,546 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690200991456 2023-07-24 12:16:36,546 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1690200991629 2023-07-24 12:16:36,547 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting 4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1690200991629 2023-07-24 12:16:36,547 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1690200991804 2023-07-24 12:16:36,547 INFO [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 6b1c90f5fec687b8c357b1ca17c402ea 2023-07-24 12:16:36,547 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1690200991804 2023-07-24 12:16:36,548 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=118, resume processing ppid=116 2023-07-24 12:16:36,548 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 21bfd26c5d6e7afe0247ea257443adaf, server=jenkins-hbase6.apache.org,34575,1690200894532 in 231 msec 2023-07-24 12:16:36,548 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690200991981 2023-07-24 12:16:36,548 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690200991981 2023-07-24 12:16:36,549 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1 2023-07-24 12:16:36,549 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1 2023-07-24 12:16:36,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=21bfd26c5d6e7afe0247ea257443adaf, ASSIGN in 392 msec 2023-07-24 12:16:36,549 INFO [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b1c90f5fec687b8c357b1ca17c402ea columnFamilyName cf1 2023-07-24 12:16:36,564 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] throttle.PressureAwareThroughputController(145): 21bfd26c5d6e7afe0247ea257443adaf#cf2#compaction#108 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:16:36,567 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] throttle.PressureAwareThroughputController(145): 21bfd26c5d6e7afe0247ea257443adaf#cf1#compaction#109 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:16:36,568 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/3d7eb053d90b448286b6e3ef5ea34c94-bottom 2023-07-24 12:16:36,574 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/4f9ffc93bda548d280a3256a5e7c3cc2-bottom 2023-07-24 12:16:36,580 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf-bottom 2023-07-24 12:16:36,584 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a18e29f739c94ff5a1d1eb3a03413310-bottom 2023-07-24 12:16:36,597 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a663a0b330824bd8a6895bb1de6c87fa-bottom 2023-07-24 12:16:36,604 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/ba5378e268374a1999f3ea6a457a558d-bottom 2023-07-24 12:16:36,604 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/.tmp/cf1/1bf4046efc1f43108326edb302927084 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/1bf4046efc1f43108326edb302927084 2023-07-24 12:16:36,605 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/.tmp/cf2/80e8519d6d3648d1bdbe682583ab396c as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/80e8519d6d3648d1bdbe682583ab396c 2023-07-24 12:16:36,609 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/dc777e780436495e8c0ea1e36ffa697b-bottom 2023-07-24 12:16:36,609 INFO [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(310): Store=6b1c90f5fec687b8c357b1ca17c402ea/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:36,609 INFO [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 6b1c90f5fec687b8c357b1ca17c402ea 2023-07-24 12:16:36,610 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 21bfd26c5d6e7afe0247ea257443adaf/cf1 of 21bfd26c5d6e7afe0247ea257443adaf into 1bf4046efc1f43108326edb302927084(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:16:36,610 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 21bfd26c5d6e7afe0247ea257443adaf: 2023-07-24 12:16:36,610 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf., storeName=21bfd26c5d6e7afe0247ea257443adaf/cf1, priority=9, startTime=1690200996538; duration=0sec 2023-07-24 12:16:36,610 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:36,610 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2 2023-07-24 12:16:36,610 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2 2023-07-24 12:16:36,611 INFO [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 6b1c90f5fec687b8c357b1ca17c402ea columnFamilyName cf2 2023-07-24 12:16:36,611 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 21bfd26c5d6e7afe0247ea257443adaf/cf2 of 21bfd26c5d6e7afe0247ea257443adaf into 80e8519d6d3648d1bdbe682583ab396c(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:16:36,611 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 21bfd26c5d6e7afe0247ea257443adaf: 2023-07-24 12:16:36,611 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf., storeName=21bfd26c5d6e7afe0247ea257443adaf/cf2, priority=9, startTime=1690200996541; duration=0sec 2023-07-24 12:16:36,611 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:36,622 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/0a871c82146d4fcda0b36e2e944ee8b4-bottom 2023-07-24 12:16:36,625 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/1f00641dc7a04e22a71975cc7596096c-bottom 2023-07-24 12:16:36,628 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/483d21f9d9e4477695a683a881b66ce6-bottom 2023-07-24 12:16:36,631 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/716242cab60c4c8283be7787fbfac46d-bottom 2023-07-24 12:16:36,633 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/731e2d7b883a49f4bda83afb824a68a1-bottom 2023-07-24 12:16:36,636 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/99f25d22508e4e5c90b523abeff3417b-bottom 2023-07-24 12:16:36,639 DEBUG [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91-bottom 2023-07-24 12:16:36,639 INFO [StoreOpener-6b1c90f5fec687b8c357b1ca17c402ea-1] regionserver.HStore(310): Store=6b1c90f5fec687b8c357b1ca17c402ea/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:36,640 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea 2023-07-24 12:16:36,641 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea 2023-07-24 12:16:36,643 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testRecoveryAndDoubleExecution descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:16:36,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 6b1c90f5fec687b8c357b1ca17c402ea 2023-07-24 12:16:36,644 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 6b1c90f5fec687b8c357b1ca17c402ea; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72730625, jitterRate=0.0837707668542862}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:16:36,644 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 6b1c90f5fec687b8c357b1ca17c402ea: 2023-07-24 12:16:36,645 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea., pid=117, masterSystemTime=1690200996463 2023-07-24 12:16:36,645 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:36,646 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:36,646 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:16:36,646 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:16:36,646 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea. 2023-07-24 12:16:36,646 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea. 2023-07-24 12:16:36,646 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1912): 6b1c90f5fec687b8c357b1ca17c402ea/cf2 is initiating minor compaction (all files) 2023-07-24 12:16:36,646 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1912): 6b1c90f5fec687b8c357b1ca17c402ea/cf1 is initiating minor compaction (all files) 2023-07-24 12:16:36,646 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 6b1c90f5fec687b8c357b1ca17c402ea/cf2 in testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea. 2023-07-24 12:16:36,646 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 6b1c90f5fec687b8c357b1ca17c402ea/cf1 in testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea. 2023-07-24 12:16:36,647 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/99f25d22508e4e5c90b523abeff3417b-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/483d21f9d9e4477695a683a881b66ce6-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/731e2d7b883a49f4bda83afb824a68a1-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/0a871c82146d4fcda0b36e2e944ee8b4-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/1f00641dc7a04e22a71975cc7596096c-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/716242cab60c4c8283be7787fbfac46d-bottom] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/.tmp, totalSize=34.9 K 2023-07-24 12:16:36,647 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/3d7eb053d90b448286b6e3ef5ea34c94-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/dc777e780436495e8c0ea1e36ffa697b-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/ba5378e268374a1999f3ea6a457a558d-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/4f9ffc93bda548d280a3256a5e7c3cc2-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a663a0b330824bd8a6895bb1de6c87fa-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a18e29f739c94ff5a1d1eb3a03413310-bottom] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/.tmp, totalSize=34.9 K 2023-07-24 12:16:36,647 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690200990910 2023-07-24 12:16:36,647 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting 9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690200990910 2023-07-24 12:16:36,648 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea. 2023-07-24 12:16:36,648 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea. 2023-07-24 12:16:36,648 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690200991073 2023-07-24 12:16:36,648 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting 3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690200991073 2023-07-24 12:16:36,648 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=115 updating hbase:meta row=6b1c90f5fec687b8c357b1ca17c402ea, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:36,648 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690200991284 2023-07-24 12:16:36,648 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690200996648"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200996648"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200996648"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200996648"}]},"ts":"1690200996648"} 2023-07-24 12:16:36,648 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690200991284 2023-07-24 12:16:36,649 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690200991456 2023-07-24 12:16:36,649 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690200991456 2023-07-24 12:16:36,649 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690200991629 2023-07-24 12:16:36,649 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting 4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690200991629 2023-07-24 12:16:36,650 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690200991804 2023-07-24 12:16:36,650 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690200991804 2023-07-24 12:16:36,650 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting 716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690200991981 2023-07-24 12:16:36,650 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690200991981 2023-07-24 12:16:36,651 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=117, resume processing ppid=115 2023-07-24 12:16:36,651 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 6b1c90f5fec687b8c357b1ca17c402ea, server=jenkins-hbase6.apache.org,34575,1690200894532 in 339 msec 2023-07-24 12:16:36,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=115, resume processing ppid=112 2023-07-24 12:16:36,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6b1c90f5fec687b8c357b1ca17c402ea, ASSIGN in 496 msec 2023-07-24 12:16:36,653 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: false 2023-07-24 12:16:36,653 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1759): TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,653 INFO [PEWorker-1] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:16:36,653 WARN [PEWorker-1] procedure2.ProcedureExecutor$WorkerThread(1980): Worker terminating UNNATURALLY null java.lang.RuntimeException: TESTING: Kill BEFORE store update: pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.kill(ProcedureExecutor.java:1761) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.execProcedure(ProcedureExecutor.java:1703) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.executeProcedure(ProcedureExecutor.java:1392) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor.access$1100(ProcedureExecutor.java:73) at org.apache.hadoop.hbase.procedure2.ProcedureExecutor$WorkerThread.run(ProcedureExecutor.java:1964) 2023-07-24 12:16:36,656 INFO [Listener at localhost/37279] procedure.MasterProcedureTestingUtility(454): Restart 10 exec state=pid=112, state=SUCCESS, locked=true; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,656 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(118): RESTART - Stop 2023-07-24 12:16:36,656 INFO [Listener at localhost/37279] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:16:36,656 INFO [Listener at localhost/37279] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:16:36,656 ERROR [Listener at localhost/37279] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] 2023-07-24 12:16:36,657 INFO [Listener at localhost/37279] procedure2.ProcedureTestingUtility(130): RESTART - Start 2023-07-24 12:16:36,657 INFO [Listener at localhost/37279] region.RegionProcedureStore(104): Starting the Region Procedure Store, number threads=1 2023-07-24 12:16:36,657 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(562): Starting 1 core workers (bigger of cpus/4 or 16) with max (burst) worker count=10 2023-07-24 12:16:36,657 INFO [Listener at localhost/37279] region.RegionProcedureStore(255): Starting Region Procedure Store lease recovery... 2023-07-24 12:16:36,657 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(582): Recovered RegionProcedureStore lease in 0 msec 2023-07-24 12:16:36,658 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=112, ppid=-1, class=org.apache.hadoop.hbase.master.assignment.SplitTableRegionProcedure) stack ids=[0, 1, 2, 7, 8, 9, 10, 11, 12, 13] 2023-07-24 12:16:36,658 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=113, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[3, 6] 2023-07-24 12:16:36,659 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=114, ppid=113, class=org.apache.hadoop.hbase.master.assignment.CloseRegionProcedure) stack ids=[4, 5] 2023-07-24 12:16:36,659 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=115, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[15, 16, 23] 2023-07-24 12:16:36,659 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=117, ppid=115, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[17, 22] 2023-07-24 12:16:36,659 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=116, ppid=112, class=org.apache.hadoop.hbase.master.assignment.TransitRegionStateProcedure) stack ids=[14, 18, 21] 2023-07-24 12:16:36,659 DEBUG [Listener at localhost/37279] store.ProcedureTree(114): Procedure Procedure(pid=118, ppid=116, class=org.apache.hadoop.hbase.master.assignment.OpenRegionProcedure) stack ids=[19, 20] 2023-07-24 12:16:36,659 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=1, state=SUCCESS; InitMetaProcedure table=hbase:meta 2023-07-24 12:16:36,660 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=4, state=SUCCESS; CreateTableProcedure table=hbase:namespace 2023-07-24 12:16:36,660 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=7, state=SUCCESS; CreateNamespaceProcedure, namespace=default 2023-07-24 12:16:36,660 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=8, state=SUCCESS; CreateNamespaceProcedure, namespace=hbase 2023-07-24 12:16:36,660 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=9, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:36,660 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=12, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionUnevenDaughter, parent=f5e0e5aefac66bded47d55c23d50c766, daughterA=7b2eccad2ce02a648b563be8a6c9f93e, daughterB=f1b51e1d4f8265994182eae453729fd8 2023-07-24 12:16:36,661 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=19, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:36,661 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=24, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionUnevenDaughter 2023-07-24 12:16:36,661 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=25, state=SUCCESS; CreateTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:36,661 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=28, state=SUCCESS; SplitTableRegionProcedure table=testSplitWithoutPONR, parent=6ee0f9264c2e2de0329ac96e30326850, daughterA=8f319344d8b0142d3e6f98cf87650169, daughterB=509695ae5946d9f03c670f89496b9662 2023-07-24 12:16:36,661 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=35, state=SUCCESS; DisableTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:36,662 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=40, state=SUCCESS; DeleteTableProcedure table=testSplitWithoutPONR 2023-07-24 12:16:36,662 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=41, state=SUCCESS; CreateTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:36,662 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=44, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.procedure2.ProcedureAbortedException via SplitTableRegionProcedure:org.apache.hadoop.hbase.procedure2.ProcedureAbortedException: abort requested; SplitTableRegionProcedure table=testRollbackAndDoubleExecution, parent=0030755a278176ce5ddf3114bdcdb65b, daughterA=9d2e4ba64e3372b74270d6f2ef4c0490, daughterB=054dd5a119895a40f5f57d00e79d7def 2023-07-24 12:16:36,662 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=47, state=SUCCESS; TransitRegionStateProcedure table=testRollbackAndDoubleExecution, region=0030755a278176ce5ddf3114bdcdb65b, ASSIGN 2023-07-24 12:16:36,662 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=49, state=SUCCESS; DisableTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:36,662 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=52, state=SUCCESS; DeleteTableProcedure table=testRollbackAndDoubleExecution 2023-07-24 12:16:36,663 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=53, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:36,663 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=56, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionNoStoreFile, parent=6ee4b5a3446811147857e7c4855e0fc5, daughterA=f5462d2626c742685e8ed96fb350e173, daughterB=868f7f720d1f1b0e94a32099a89c6854 2023-07-24 12:16:36,663 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=63, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:36,663 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=68, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionNoStoreFile 2023-07-24 12:16:36,663 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=69, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:36,663 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=72, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionDeletedRowsDaughter, parent=ed62c139047156182f9c4cdbe34edfa7, daughterA=d6d1a02c2c1c2259a534e0cfc7642c20, daughterB=7d24409abbbbe6199617e8fdd3054481 2023-07-24 12:16:36,663 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=79, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:36,663 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=84, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionDeletedRowsDaughter 2023-07-24 12:16:36,664 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=85, state=SUCCESS; CreateTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:36,664 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=88, state=SUCCESS; ModifyTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:36,664 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=95, state=ROLLEDBACK, exception=org.apache.hadoop.hbase.HBaseIOException via master-split-regions:org.apache.hadoop.hbase.HBaseIOException: The parent region state=OPENING, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRollbackForSplitTableRegionWithReplica, region=2e4f3d10799c74f83e99aa427a3ad520 is currently in transition, give up; SplitTableRegionProcedure table=testRollbackForSplitTableRegionWithReplica, parent=01a858500fdab441704f5de8ae140206, daughterA=433461289b4d9e26486169c0e4440c27, daughterB=f86a8946d555c170556384d277c059ea 2023-07-24 12:16:36,664 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=96, state=SUCCESS; DisableTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:36,664 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] throttle.PressureAwareThroughputController(145): 6b1c90f5fec687b8c357b1ca17c402ea#cf2#compaction#110 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:16:36,664 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=101, state=SUCCESS; DeleteTableProcedure table=testRollbackForSplitTableRegionWithReplica 2023-07-24 12:16:36,665 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] throttle.PressureAwareThroughputController(145): 6b1c90f5fec687b8c357b1ca17c402ea#cf1#compaction#111 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:16:36,665 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=102, state=SUCCESS; CreateTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:36,665 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=105, state=SUCCESS; DisableTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:36,665 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=108, state=SUCCESS; DeleteTableProcedure table=testInvalidSplitKey 2023-07-24 12:16:36,665 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(411): Completed pid=109, state=SUCCESS; CreateTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:36,667 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,667 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN 2023-07-24 12:16:36,667 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:36,667 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6b1c90f5fec687b8c357b1ca17c402ea, ASSIGN 2023-07-24 12:16:36,667 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=21bfd26c5d6e7afe0247ea257443adaf, ASSIGN 2023-07-24 12:16:36,667 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 6b1c90f5fec687b8c357b1ca17c402ea, server=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:36,667 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(468): Loading pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 21bfd26c5d6e7afe0247ea257443adaf, server=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:36,667 DEBUG [Listener at localhost/37279] procedure2.Procedure(966): pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf held the lock before restarting, call acquireLock to restore it. 2023-07-24 12:16:36,667 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,667 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,667 INFO [Listener at localhost/37279] procedure.MasterProcedureScheduler(727): Took xlock for pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,667 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=113, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141, UNASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:36,667 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=114, ppid=113, state=SUCCESS; CloseRegionProcedure d8d7c0b2b9c0adab4dd247a24a5b7141, server=jenkins-hbase6.apache.org,34575,1690200894532 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:36,667 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=115, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6b1c90f5fec687b8c357b1ca17c402ea, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:36,667 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=116, ppid=112, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=21bfd26c5d6e7afe0247ea257443adaf, ASSIGN didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:36,667 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=117, ppid=115, state=SUCCESS; OpenRegionProcedure 6b1c90f5fec687b8c357b1ca17c402ea, server=jenkins-hbase6.apache.org,34575,1690200894532 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:36,668 DEBUG [Listener at localhost/37279] procedure2.Procedure(946): pid=118, ppid=116, state=SUCCESS; OpenRegionProcedure 21bfd26c5d6e7afe0247ea257443adaf, server=jenkins-hbase6.apache.org,34575,1690200894532 didn't hold the lock before restarting, skip acquiring lock. 2023-07-24 12:16:36,668 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(596): Loaded RegionProcedureStore in 10 msec 2023-07-24 12:16:36,670 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(253): hbase:meta replica znodes: [meta-region-server] 2023-07-24 12:16:36,670 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(272): Loaded hbase:meta state=OPEN, location=jenkins-hbase6.apache.org,41789,1690200894488, table=hbase:meta, region=1588230740 2023-07-24 12:16:36,675 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.procedure2.CompletedProcedureCleaner; timeout=30000, timestamp=1690201026675 2023-07-24 12:16:36,680 DEBUG [Listener at localhost/37279] assignment.AssignmentManager(1517): Joining cluster... 2023-07-24 12:16:36,681 DEBUG [PEWorker-1] procedure2.ProcedureExecutor(1400): LOCK_EVENT_WAIT pid=112, state=RUNNABLE:SPLIT_TABLE_REGION_POST_OPERATION; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:36,684 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=f3d7ba30038785f88ca0646c63b6c13d, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:36,684 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=d8d7c0b2b9c0adab4dd247a24a5b7141, regionState=CLOSED, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=2 2023-07-24 12:16:36,684 INFO [Listener at localhost/37279] assignment.RegionStates(657): Added to offline, CURRENTLY NEVER CLEARED!!! state=CLOSED, location=jenkins-hbase6.apache.org,34575,1690200894532, table=testRecoveryAndDoubleExecution, region=d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:36,684 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=6b1c90f5fec687b8c357b1ca17c402ea, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=86 2023-07-24 12:16:36,685 DEBUG [Listener at localhost/37279] assignment.RegionStateStore(147): Load hbase:meta entry region=21bfd26c5d6e7afe0247ea257443adaf, regionState=OPEN, lastHost=jenkins-hbase6.apache.org,34575,1690200894532, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532, openSeqNum=86 2023-07-24 12:16:36,685 INFO [Listener at localhost/37279] assignment.AssignmentManager(1529): Number of RegionServers=3 2023-07-24 12:16:36,685 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$RegionInTransitionChore; timeout=60000, timestamp=1690201056685 2023-07-24 12:16:36,685 INFO [Listener at localhost/37279] procedure2.TimeoutExecutorThread(81): ADDED pid=-1, state=WAITING_TIMEOUT; org.apache.hadoop.hbase.master.assignment.AssignmentManager$DeadServerMetricRegionChore; timeout=120000, timestamp=1690201116685 2023-07-24 12:16:36,685 INFO [Listener at localhost/37279] assignment.AssignmentManager(1536): Joined the cluster in 5 msec 2023-07-24 12:16:36,685 WARN [PEWorker-1] procedure2.ProcedureExecutor$Testing(122): Toggle KILL before store update to: true 2023-07-24 12:16:36,689 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=112, state=SUCCESS; SplitTableRegionProcedure table=testRecoveryAndDoubleExecution, parent=d8d7c0b2b9c0adab4dd247a24a5b7141, daughterA=6b1c90f5fec687b8c357b1ca17c402ea, daughterB=21bfd26c5d6e7afe0247ea257443adaf in 4.4450 sec 2023-07-24 12:16:36,694 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/.tmp/cf2/d424d09a31514003ad3d4297c3b717a8 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/d424d09a31514003ad3d4297c3b717a8 2023-07-24 12:16:36,694 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/.tmp/cf1/dd70793527684935a3e72db1b50cd6d6 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/dd70793527684935a3e72db1b50cd6d6 2023-07-24 12:16:36,698 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6b1c90f5fec687b8c357b1ca17c402ea/cf2 of 6b1c90f5fec687b8c357b1ca17c402ea into d424d09a31514003ad3d4297c3b717a8(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:16:36,698 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 6b1c90f5fec687b8c357b1ca17c402ea/cf1 of 6b1c90f5fec687b8c357b1ca17c402ea into dd70793527684935a3e72db1b50cd6d6(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:16:36,698 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6b1c90f5fec687b8c357b1ca17c402ea: 2023-07-24 12:16:36,698 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 6b1c90f5fec687b8c357b1ca17c402ea: 2023-07-24 12:16:36,698 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea., storeName=6b1c90f5fec687b8c357b1ca17c402ea/cf2, priority=9, startTime=1690200996646; duration=0sec 2023-07-24 12:16:36,698 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea., storeName=6b1c90f5fec687b8c357b1ca17c402ea/cf1, priority=9, startTime=1690200996645; duration=0sec 2023-07-24 12:16:36,698 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:36,698 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:36,936 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34575] ipc.CallRunner(144): callId: 856 service: ClientService methodName: Scan size: 154 connection: 172.31.2.20:46684 deadline: 1690201056936, exception=org.apache.hadoop.hbase.NotServingRegionException: testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141. is not online on jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:37,049 DEBUG [Listener at localhost/37279] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-24 12:16:37,054 INFO [Listener at localhost/37279] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-24 12:16:37,055 DEBUG [Listener at localhost/37279] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-24 12:16:37,068 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:16:37,069 INFO [Listener at localhost/37279] client.HBaseAdmin$15(890): Started disable of testRecoveryAndDoubleExecution 2023-07-24 12:16:37,070 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$11(2418): Client=jenkins//172.31.2.20 disable testRecoveryAndDoubleExecution 2023-07-24 12:16:37,070 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=119, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:37,073 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-07-24 12:16:37,073 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200997073"}]},"ts":"1690200997073"} 2023-07-24 12:16:37,074 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLING in hbase:meta 2023-07-24 12:16:37,076 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testRecoveryAndDoubleExecution to state=DISABLING 2023-07-24 12:16:37,077 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6b1c90f5fec687b8c357b1ca17c402ea, UNASSIGN}, {pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=21bfd26c5d6e7afe0247ea257443adaf, UNASSIGN}] 2023-07-24 12:16:37,077 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=121, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=21bfd26c5d6e7afe0247ea257443adaf, UNASSIGN 2023-07-24 12:16:37,078 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=21bfd26c5d6e7afe0247ea257443adaf, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:37,078 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690200997078"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200997078"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200997078"}]},"ts":"1690200997078"} 2023-07-24 12:16:37,079 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=122, ppid=121, state=RUNNABLE; CloseRegionProcedure 21bfd26c5d6e7afe0247ea257443adaf, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:37,081 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=120, ppid=119, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6b1c90f5fec687b8c357b1ca17c402ea, UNASSIGN 2023-07-24 12:16:37,082 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=6b1c90f5fec687b8c357b1ca17c402ea, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:37,082 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690200997082"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200997082"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200997082"}]},"ts":"1690200997082"} 2023-07-24 12:16:37,083 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=123, ppid=120, state=RUNNABLE; CloseRegionProcedure 6b1c90f5fec687b8c357b1ca17c402ea, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:37,104 ERROR [BP-51740533-172.31.2.20-1690200889613 heartbeating to localhost/127.0.0.1:33821] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-51740533-172.31.2.20-1690200889613 (Datanode Uuid 8ac734ec-00fd-4136-ae08-8d364bd1fe4b) service to localhost/127.0.0.1:33821 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-24 12:16:37,174 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-07-24 12:16:37,231 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:37,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 21bfd26c5d6e7afe0247ea257443adaf, disabling compactions & flushes 2023-07-24 12:16:37,231 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf. 2023-07-24 12:16:37,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf. 2023-07-24 12:16:37,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf. after waiting 0 ms 2023-07-24 12:16:37,231 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf. 2023-07-24 12:16:37,232 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/3d7eb053d90b448286b6e3ef5ea34c94-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/dc777e780436495e8c0ea1e36ffa697b-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/ba5378e268374a1999f3ea6a457a558d-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/4f9ffc93bda548d280a3256a5e7c3cc2-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a663a0b330824bd8a6895bb1de6c87fa-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a18e29f739c94ff5a1d1eb3a03413310-top] to archive 2023-07-24 12:16:37,233 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:16:37,235 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,236 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,237 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,237 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,238 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,239 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,240 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,241 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/99f25d22508e4e5c90b523abeff3417b-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/483d21f9d9e4477695a683a881b66ce6-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/731e2d7b883a49f4bda83afb824a68a1-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/0a871c82146d4fcda0b36e2e944ee8b4-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/1f00641dc7a04e22a71975cc7596096c-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/716242cab60c4c8283be7787fbfac46d-top] to archive 2023-07-24 12:16:37,242 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:16:37,243 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,244 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,245 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,246 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,247 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,249 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,250 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,253 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 12:16:37,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:16:37,254 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf. 2023-07-24 12:16:37,254 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 21bfd26c5d6e7afe0247ea257443adaf: 2023-07-24 12:16:37,256 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:37,256 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 6b1c90f5fec687b8c357b1ca17c402ea 2023-07-24 12:16:37,256 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 6b1c90f5fec687b8c357b1ca17c402ea, disabling compactions & flushes 2023-07-24 12:16:37,257 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea. 2023-07-24 12:16:37,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea. 2023-07-24 12:16:37,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea. after waiting 0 ms 2023-07-24 12:16:37,257 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea. 2023-07-24 12:16:37,257 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=121 updating hbase:meta row=21bfd26c5d6e7afe0247ea257443adaf, regionState=CLOSED 2023-07-24 12:16:37,257 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690200997257"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200997257"}]},"ts":"1690200997257"} 2023-07-24 12:16:37,258 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/3d7eb053d90b448286b6e3ef5ea34c94-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/dc777e780436495e8c0ea1e36ffa697b-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/ba5378e268374a1999f3ea6a457a558d-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/4f9ffc93bda548d280a3256a5e7c3cc2-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a663a0b330824bd8a6895bb1de6c87fa-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a18e29f739c94ff5a1d1eb3a03413310-bottom] to archive 2023-07-24 12:16:37,259 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:16:37,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=122, resume processing ppid=121 2023-07-24 12:16:37,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=122, ppid=121, state=SUCCESS; CloseRegionProcedure 21bfd26c5d6e7afe0247ea257443adaf, server=jenkins-hbase6.apache.org,34575,1690200894532 in 180 msec 2023-07-24 12:16:37,261 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,262 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=121, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=21bfd26c5d6e7afe0247ea257443adaf, UNASSIGN in 184 msec 2023-07-24 12:16:37,263 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,264 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,265 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,265 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,266 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,267 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,269 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/99f25d22508e4e5c90b523abeff3417b-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/483d21f9d9e4477695a683a881b66ce6-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/731e2d7b883a49f4bda83afb824a68a1-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/0a871c82146d4fcda0b36e2e944ee8b4-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/1f00641dc7a04e22a71975cc7596096c-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/716242cab60c4c8283be7787fbfac46d-bottom] to archive 2023-07-24 12:16:37,269 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:16:37,270 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,271 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,272 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,273 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,274 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,275 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,276 DEBUG [StoreCloser-testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 12:16:37,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:16:37,279 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea. 2023-07-24 12:16:37,279 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 6b1c90f5fec687b8c357b1ca17c402ea: 2023-07-24 12:16:37,281 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 6b1c90f5fec687b8c357b1ca17c402ea 2023-07-24 12:16:37,281 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=120 updating hbase:meta row=6b1c90f5fec687b8c357b1ca17c402ea, regionState=CLOSED 2023-07-24 12:16:37,281 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.","families":{"info":[{"qualifier":"regioninfo","vlen":66,"tag":[],"timestamp":"1690200997281"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200997281"}]},"ts":"1690200997281"} 2023-07-24 12:16:37,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=123, resume processing ppid=120 2023-07-24 12:16:37,284 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=123, ppid=120, state=SUCCESS; CloseRegionProcedure 6b1c90f5fec687b8c357b1ca17c402ea, server=jenkins-hbase6.apache.org,34575,1690200894532 in 199 msec 2023-07-24 12:16:37,285 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=120, resume processing ppid=119 2023-07-24 12:16:37,285 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=120, ppid=119, state=SUCCESS; TransitRegionStateProcedure table=testRecoveryAndDoubleExecution, region=6b1c90f5fec687b8c357b1ca17c402ea, UNASSIGN in 207 msec 2023-07-24 12:16:37,285 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200997285"}]},"ts":"1690200997285"} 2023-07-24 12:16:37,286 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testRecoveryAndDoubleExecution, state=DISABLED in hbase:meta 2023-07-24 12:16:37,287 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testRecoveryAndDoubleExecution to state=DISABLED 2023-07-24 12:16:37,289 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=119, state=SUCCESS; DisableTableProcedure table=testRecoveryAndDoubleExecution in 218 msec 2023-07-24 12:16:37,375 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=119 2023-07-24 12:16:37,375 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testRecoveryAndDoubleExecution, procId: 119 completed 2023-07-24 12:16:37,376 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$5(2228): Client=jenkins//172.31.2.20 delete testRecoveryAndDoubleExecution 2023-07-24 12:16:37,377 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:37,378 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=124, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:37,379 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-07-24 12:16:37,379 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=124, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:37,384 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,384 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea 2023-07-24 12:16:37,384 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:37,386 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/recovered.edits] 2023-07-24 12:16:37,387 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/recovered.edits] 2023-07-24 12:16:37,387 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/recovered.edits] 2023-07-24 12:16:37,392 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/.regioninfo to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/.regioninfo 2023-07-24 12:16:37,393 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/3d7eb053d90b448286b6e3ef5ea34c94 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/3d7eb053d90b448286b6e3ef5ea34c94 2023-07-24 12:16:37,394 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/.regioninfo to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/.regioninfo 2023-07-24 12:16:37,395 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/4f9ffc93bda548d280a3256a5e7c3cc2 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/4f9ffc93bda548d280a3256a5e7c3cc2 2023-07-24 12:16:37,396 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf 2023-07-24 12:16:37,397 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf1/3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf1/3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,397 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf1/3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf1/3d7eb053d90b448286b6e3ef5ea34c94.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,398 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a18e29f739c94ff5a1d1eb3a03413310 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a18e29f739c94ff5a1d1eb3a03413310 2023-07-24 12:16:37,399 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf1/4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf1/4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,400 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf1/4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf1/4f9ffc93bda548d280a3256a5e7c3cc2.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,400 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a663a0b330824bd8a6895bb1de6c87fa to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/a663a0b330824bd8a6895bb1de6c87fa 2023-07-24 12:16:37,401 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,402 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf1/9c9e0fcff6d64de2aaab1d34781d9dbf.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,402 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/ba5378e268374a1999f3ea6a457a558d to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/ba5378e268374a1999f3ea6a457a558d 2023-07-24 12:16:37,403 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf1/a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf1/a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,404 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf1/a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf1/a18e29f739c94ff5a1d1eb3a03413310.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,404 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/dc777e780436495e8c0ea1e36ffa697b to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf1/dc777e780436495e8c0ea1e36ffa697b 2023-07-24 12:16:37,404 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf1/a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf1/a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,405 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf1/a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf1/a663a0b330824bd8a6895bb1de6c87fa.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,406 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf1/ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf1/ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,407 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf1/ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf1/ba5378e268374a1999f3ea6a457a558d.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,407 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf1/dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf1/dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,407 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/0a871c82146d4fcda0b36e2e944ee8b4 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/0a871c82146d4fcda0b36e2e944ee8b4 2023-07-24 12:16:37,408 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf1/dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf1/dc777e780436495e8c0ea1e36ffa697b.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,410 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/1f00641dc7a04e22a71975cc7596096c to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/1f00641dc7a04e22a71975cc7596096c 2023-07-24 12:16:37,410 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf2/0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf2/0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,413 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/483d21f9d9e4477695a683a881b66ce6 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/483d21f9d9e4477695a683a881b66ce6 2023-07-24 12:16:37,413 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf2/1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf2/1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,414 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf2/0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf2/0a871c82146d4fcda0b36e2e944ee8b4.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,414 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/716242cab60c4c8283be7787fbfac46d to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/716242cab60c4c8283be7787fbfac46d 2023-07-24 12:16:37,415 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf2/483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf2/483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,415 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf2/1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf2/1f00641dc7a04e22a71975cc7596096c.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,416 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/731e2d7b883a49f4bda83afb824a68a1 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/731e2d7b883a49f4bda83afb824a68a1 2023-07-24 12:16:37,416 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf2/716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf2/716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,417 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf2/483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf2/483d21f9d9e4477695a683a881b66ce6.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,417 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf2/731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf2/731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,417 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/99f25d22508e4e5c90b523abeff3417b to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/99f25d22508e4e5c90b523abeff3417b 2023-07-24 12:16:37,418 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf2/716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf2/716242cab60c4c8283be7787fbfac46d.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,419 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf2/99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf2/99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,419 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91 2023-07-24 12:16:37,420 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf2/731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf2/731e2d7b883a49f4bda83afb824a68a1.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,420 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/6b1c90f5fec687b8c357b1ca17c402ea/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,421 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf2/99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf2/99f25d22508e4e5c90b523abeff3417b.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,421 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/recovered.edits/85.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141/recovered.edits/85.seqid 2023-07-24 12:16:37,422 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,423 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/dd70793527684935a3e72db1b50cd6d6 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf1/dd70793527684935a3e72db1b50cd6d6 2023-07-24 12:16:37,423 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/21bfd26c5d6e7afe0247ea257443adaf/cf2/eade56e0e7fa4ebe97b5eb49d1cdef91.d8d7c0b2b9c0adab4dd247a24a5b7141 2023-07-24 12:16:37,425 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/d424d09a31514003ad3d4297c3b717a8 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/cf2/d424d09a31514003ad3d4297c3b717a8 2023-07-24 12:16:37,425 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/1bf4046efc1f43108326edb302927084 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf1/1bf4046efc1f43108326edb302927084 2023-07-24 12:16:37,427 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/80e8519d6d3648d1bdbe682583ab396c to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/cf2/80e8519d6d3648d1bdbe682583ab396c 2023-07-24 12:16:37,427 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/recovered.edits/90.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea/recovered.edits/90.seqid 2023-07-24 12:16:37,428 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/6b1c90f5fec687b8c357b1ca17c402ea 2023-07-24 12:16:37,429 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/recovered.edits/90.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf/recovered.edits/90.seqid 2023-07-24 12:16:37,430 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testRecoveryAndDoubleExecution/21bfd26c5d6e7afe0247ea257443adaf 2023-07-24 12:16:37,430 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testRecoveryAndDoubleExecution regions 2023-07-24 12:16:37,432 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=124, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:37,433 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testRecoveryAndDoubleExecution from hbase:meta 2023-07-24 12:16:37,435 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testRecoveryAndDoubleExecution' descriptor. 2023-07-24 12:16:37,436 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=124, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:37,436 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testRecoveryAndDoubleExecution' from region states. 2023-07-24 12:16:37,436 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200997436"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:37,436 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200997436"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:37,436 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690200997436"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:37,438 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-24 12:16:37,438 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => d8d7c0b2b9c0adab4dd247a24a5b7141, NAME => 'testRecoveryAndDoubleExecution,,1690200990406.d8d7c0b2b9c0adab4dd247a24a5b7141.', STARTKEY => '', ENDKEY => '', OFFLINE => true, SPLIT => true}, {ENCODED => 6b1c90f5fec687b8c357b1ca17c402ea, NAME => 'testRecoveryAndDoubleExecution,,1690200992240.6b1c90f5fec687b8c357b1ca17c402ea.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 21bfd26c5d6e7afe0247ea257443adaf, NAME => 'testRecoveryAndDoubleExecution,41,1690200992240.21bfd26c5d6e7afe0247ea257443adaf.', STARTKEY => '41', ENDKEY => ''}] 2023-07-24 12:16:37,438 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testRecoveryAndDoubleExecution' as deleted. 2023-07-24 12:16:37,438 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testRecoveryAndDoubleExecution","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690200997438"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:37,439 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testRecoveryAndDoubleExecution state from META 2023-07-24 12:16:37,441 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=124, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testRecoveryAndDoubleExecution 2023-07-24 12:16:37,442 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=124, state=SUCCESS; DeleteTableProcedure table=testRecoveryAndDoubleExecution in 65 msec 2023-07-24 12:16:37,480 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=124 2023-07-24 12:16:37,480 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testRecoveryAndDoubleExecution, procId: 124 completed 2023-07-24 12:16:37,495 INFO [Listener at localhost/37279] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testRecoveryAndDoubleExecution Thread=377 (was 371) - Thread LEAK? -, OpenFileDescriptor=841 (was 750) - OpenFileDescriptor LEAK? -, MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=175 (was 182), ProcessCount=168 (was 168), AvailableMemoryMB=1489 (was 1545) 2023-07-24 12:16:37,508 INFO [Listener at localhost/37279] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=377, OpenFileDescriptor=841, MaxFileDescriptor=60000, SystemLoadAverage=175, ProcessCount=168, AvailableMemoryMB=1488 2023-07-24 12:16:37,508 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:16:37,510 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 12:16:37,510 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(492): Client=jenkins//172.31.2.20 set balanceSwitch=false 2023-07-24 12:16:37,511 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 12:16:37,513 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 12:16:37,513 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 12:16:37,515 DEBUG [HFileArchiver-16] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:37,515 DEBUG [HFileArchiver-16] backup.HFileArchiver(153): Directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76 empty. 2023-07-24 12:16:37,516 DEBUG [HFileArchiver-16] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:37,516 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-07-24 12:16:37,924 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/.tabledesc/.tableinfo.0000000001 2023-07-24 12:16:37,926 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(7675): creating {ENCODED => ff6186d783435ac95747fe76928d7a76, NAME => 'testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegionEmptyDaughter', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp 2023-07-24 12:16:37,933 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:16:37,933 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1604): Closing ff6186d783435ac95747fe76928d7a76, disabling compactions & flushes 2023-07-24 12:16:37,933 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:37,933 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:37,933 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. after waiting 0 ms 2023-07-24 12:16:37,933 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:37,933 INFO [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:37,933 DEBUG [RegionOpenAndInit-testSplitTableRegionEmptyDaughter-pool-0] regionserver.HRegion(1558): Region close journal for ff6186d783435ac95747fe76928d7a76: 2023-07-24 12:16:37,935 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 12:16:37,936 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690200997935"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690200997935"}]},"ts":"1690200997935"} 2023-07-24 12:16:37,937 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 12:16:37,937 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 12:16:37,937 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200997937"}]},"ts":"1690200997937"} 2023-07-24 12:16:37,938 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLING in hbase:meta 2023-07-24 12:16:37,941 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase6.apache.org=0} racks are {/default-rack=0} 2023-07-24 12:16:37,942 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 12:16:37,942 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 12:16:37,942 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 12:16:37,942 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 12:16:37,943 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=ff6186d783435ac95747fe76928d7a76, ASSIGN}] 2023-07-24 12:16:37,944 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=ff6186d783435ac95747fe76928d7a76, ASSIGN 2023-07-24 12:16:37,944 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=126, ppid=125, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=ff6186d783435ac95747fe76928d7a76, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,34575,1690200894532; forceNewPlan=false, retain=false 2023-07-24 12:16:38,095 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 12:16:38,096 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=ff6186d783435ac95747fe76928d7a76, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,096 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690200998096"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690200998096"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690200998096"}]},"ts":"1690200998096"} 2023-07-24 12:16:38,097 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=127, ppid=126, state=RUNNABLE; OpenRegionProcedure ff6186d783435ac95747fe76928d7a76, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:38,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:38,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => ff6186d783435ac95747fe76928d7a76, NAME => 'testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76.', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:16:38,251 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:16:38,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:38,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:16:38,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:38,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:38,253 INFO [StoreOpener-ff6186d783435ac95747fe76928d7a76-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:38,254 DEBUG [StoreOpener-ff6186d783435ac95747fe76928d7a76-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1 2023-07-24 12:16:38,254 DEBUG [StoreOpener-ff6186d783435ac95747fe76928d7a76-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1 2023-07-24 12:16:38,254 INFO [StoreOpener-ff6186d783435ac95747fe76928d7a76-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ff6186d783435ac95747fe76928d7a76 columnFamilyName cf1 2023-07-24 12:16:38,255 INFO [StoreOpener-ff6186d783435ac95747fe76928d7a76-1] regionserver.HStore(310): Store=ff6186d783435ac95747fe76928d7a76/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:38,255 INFO [StoreOpener-ff6186d783435ac95747fe76928d7a76-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:38,256 DEBUG [StoreOpener-ff6186d783435ac95747fe76928d7a76-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2 2023-07-24 12:16:38,256 DEBUG [StoreOpener-ff6186d783435ac95747fe76928d7a76-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2 2023-07-24 12:16:38,256 INFO [StoreOpener-ff6186d783435ac95747fe76928d7a76-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region ff6186d783435ac95747fe76928d7a76 columnFamilyName cf2 2023-07-24 12:16:38,257 INFO [StoreOpener-ff6186d783435ac95747fe76928d7a76-1] regionserver.HStore(310): Store=ff6186d783435ac95747fe76928d7a76/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:38,257 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:38,258 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:38,259 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:16:38,260 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:38,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 12:16:38,262 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened ff6186d783435ac95747fe76928d7a76; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65723153, jitterRate=-0.02064870297908783}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:16:38,262 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for ff6186d783435ac95747fe76928d7a76: 2023-07-24 12:16:38,263 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76., pid=127, masterSystemTime=1690200998249 2023-07-24 12:16:38,264 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:38,264 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:38,265 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=126 updating hbase:meta row=ff6186d783435ac95747fe76928d7a76, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,265 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690200998265"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690200998265"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690200998265"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690200998265"}]},"ts":"1690200998265"} 2023-07-24 12:16:38,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=127, resume processing ppid=126 2023-07-24 12:16:38,268 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=127, ppid=126, state=SUCCESS; OpenRegionProcedure ff6186d783435ac95747fe76928d7a76, server=jenkins-hbase6.apache.org,34575,1690200894532 in 169 msec 2023-07-24 12:16:38,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=126, resume processing ppid=125 2023-07-24 12:16:38,269 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=126, ppid=125, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=ff6186d783435ac95747fe76928d7a76, ASSIGN in 325 msec 2023-07-24 12:16:38,270 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 12:16:38,270 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690200998270"}]},"ts":"1690200998270"} 2023-07-24 12:16:38,271 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=ENABLED in hbase:meta 2023-07-24 12:16:38,273 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=125, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegionEmptyDaughter execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 12:16:38,274 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=125, state=SUCCESS; CreateTableProcedure table=testSplitTableRegionEmptyDaughter in 762 msec 2023-07-24 12:16:38,519 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:38,521 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,521 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-24 12:16:38,521 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:38,521 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-24 12:16:38,521 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-24 12:16:38,521 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,521 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,522 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,522 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,522 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,522 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,522 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,522 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,523 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,523 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,523 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:38,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,523 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,523 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:38,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:38,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:38,524 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,524 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,524 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:38,524 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:38,524 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,524 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:38,524 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:38,524 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:38,524 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:38,524 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:38,524 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:38,525 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,525 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:38,525 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:38,525 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:38,525 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:38,525 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:38,525 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:38,525 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:38,525 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:38,525 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. started... 2023-07-24 12:16:38,526 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff6186d783435ac95747fe76928d7a76 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 12:16:38,526 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,527 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,527 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,527 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:38,528 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,528 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:38,535 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf1/ee727bc1f2df45f0b1b6651003035058 2023-07-24 12:16:38,545 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf2/f354c1248fe94d4e95b7deb9a135f78b 2023-07-24 12:16:38,548 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf1/ee727bc1f2df45f0b1b6651003035058 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/ee727bc1f2df45f0b1b6651003035058 2023-07-24 12:16:38,551 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/ee727bc1f2df45f0b1b6651003035058, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:16:38,552 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf2/f354c1248fe94d4e95b7deb9a135f78b as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/f354c1248fe94d4e95b7deb9a135f78b 2023-07-24 12:16:38,554 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/f354c1248fe94d4e95b7deb9a135f78b, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:16:38,555 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for ff6186d783435ac95747fe76928d7a76 in 29ms, sequenceid=6, compaction requested=false 2023-07-24 12:16:38,555 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegionEmptyDaughter' 2023-07-24 12:16:38,556 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff6186d783435ac95747fe76928d7a76: 2023-07-24 12:16:38,556 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:38,556 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:38,556 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:38,556 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,556 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:38,556 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:38,557 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,557 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,557 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:38,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:38,558 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,558 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:38,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:38,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:38,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,558 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,559 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:38,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-24 12:16:38,559 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@304c5ad5[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:38,559 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-24 12:16:38,559 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,560 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,560 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,560 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,560 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,561 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:38,561 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,561 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:38,561 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,561 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:38,561 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,561 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:38,561 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:38,561 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:38,561 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:38,561 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:38,561 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:38,561 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:38,563 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:38,563 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,563 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:38,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:38,563 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:38,563 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:38,564 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:38,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:38,564 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:38,564 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:38,564 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:38,564 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:38,564 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:38,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:38,564 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:38,565 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,566 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,567 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:38,567 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:16:38,567 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-24 12:16:38,567 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:38,567 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:38,567 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,567 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:38,568 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,568 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,568 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:38,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:38,568 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,568 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,568 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,568 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,568 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,569 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:38,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,569 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,569 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:38,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,570 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:38,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,571 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,571 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,572 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,572 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,572 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,578 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,578 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,578 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,578 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,578 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,578 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,578 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,578 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,578 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-24 12:16:38,578 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:38,578 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:38,578 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,579 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,578 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,579 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,578 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,579 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,579 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:38,579 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,579 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:38,579 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,579 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,579 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,579 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,579 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,579 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,679 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:38,680 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:38,691 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:38,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:38,692 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,692 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-24 12:16:38,692 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:38,692 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-24 12:16:38,692 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-24 12:16:38,692 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,693 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,694 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,694 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,694 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,694 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,694 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,694 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,694 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,694 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:38,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,695 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:38,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:38,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:38,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,695 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,695 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,695 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,695 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,695 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:38,695 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,695 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:38,695 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:38,695 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:38,696 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,696 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:38,696 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:38,696 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:38,696 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,696 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:38,696 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:38,696 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:38,696 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:38,696 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:38,696 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:38,696 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:38,696 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:38,696 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:38,696 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:38,696 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:38,696 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:38,696 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. started... 2023-07-24 12:16:38,696 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff6186d783435ac95747fe76928d7a76 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:38,697 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,698 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,698 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:38,698 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,698 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,698 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:38,704 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf1/70cc4e48847f4ac88d8c9565e1ae6ee6 2023-07-24 12:16:38,713 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf2/7f091c2de0c64121852ad00f8a4c34f7 2023-07-24 12:16:38,717 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf1/70cc4e48847f4ac88d8c9565e1ae6ee6 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/70cc4e48847f4ac88d8c9565e1ae6ee6 2023-07-24 12:16:38,720 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/70cc4e48847f4ac88d8c9565e1ae6ee6, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:16:38,721 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf2/7f091c2de0c64121852ad00f8a4c34f7 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/7f091c2de0c64121852ad00f8a4c34f7 2023-07-24 12:16:38,724 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/7f091c2de0c64121852ad00f8a4c34f7, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:16:38,725 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff6186d783435ac95747fe76928d7a76 in 29ms, sequenceid=19, compaction requested=false 2023-07-24 12:16:38,725 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff6186d783435ac95747fe76928d7a76: 2023-07-24 12:16:38,725 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:38,725 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:38,725 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:38,725 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,725 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:38,725 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:38,727 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,727 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:38,727 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:38,727 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,728 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:38,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:38,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:38,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,728 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,729 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:38,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-24 12:16:38,729 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@2fb2aee0[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:38,729 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-24 12:16:38,729 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,730 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,730 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,730 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,731 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,731 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:38,731 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:38,730 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,731 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:38,731 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:38,731 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,731 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,731 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:38,731 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:38,731 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:38,731 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:38,731 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:38,731 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:38,732 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:38,732 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:38,732 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,732 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:38,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:38,732 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:38,733 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:38,733 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:38,733 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:38,734 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 12:16:38,736 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:38,736 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:38,736 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:38,736 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,737 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:38,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,741 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:38,742 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:16:38,742 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-24 12:16:38,742 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:38,742 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:38,742 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,742 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:38,743 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,743 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,743 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,743 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,743 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,743 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:38,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,743 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:38,744 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,743 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:38,744 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,744 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,744 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,745 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:38,745 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,745 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,746 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:38,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,747 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,747 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,747 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,750 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,750 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,750 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,750 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-24 12:16:38,750 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:38,750 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,750 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:38,750 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,750 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,751 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:38,751 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:38,750 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,750 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,751 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,751 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,751 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,751 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,751 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,751 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,751 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,751 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,752 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,752 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,851 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:38,852 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:38,863 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:38,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:38,864 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,864 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-24 12:16:38,864 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:38,865 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-24 12:16:38,865 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-24 12:16:38,865 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,865 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,866 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,866 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,866 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,866 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,866 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,866 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,867 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,867 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:38,867 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,867 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:38,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,868 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:38,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:38,868 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,868 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,868 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:38,868 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,868 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:38,868 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:38,868 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:38,868 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,868 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,868 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:38,868 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:38,868 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:38,868 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:38,869 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:38,868 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:38,869 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:38,869 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,869 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:38,869 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:38,869 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:38,869 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:38,869 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:38,869 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:38,869 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:38,869 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:38,869 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. started... 2023-07-24 12:16:38,869 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff6186d783435ac95747fe76928d7a76 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:38,876 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,877 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,877 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,877 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:38,877 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,877 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:38,884 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf1/763fd36339d04267834a48b189e7c242 2023-07-24 12:16:38,897 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf2/7e6c82f848bc414ca58e79844e14cf13 2023-07-24 12:16:38,901 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf1/763fd36339d04267834a48b189e7c242 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/763fd36339d04267834a48b189e7c242 2023-07-24 12:16:38,905 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/763fd36339d04267834a48b189e7c242, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:16:38,905 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf2/7e6c82f848bc414ca58e79844e14cf13 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/7e6c82f848bc414ca58e79844e14cf13 2023-07-24 12:16:38,908 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/7e6c82f848bc414ca58e79844e14cf13, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:16:38,909 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff6186d783435ac95747fe76928d7a76 in 40ms, sequenceid=32, compaction requested=true 2023-07-24 12:16:38,909 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff6186d783435ac95747fe76928d7a76: 2023-07-24 12:16:38,909 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:38,909 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:38,909 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:38,909 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,909 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:38,910 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:38,913 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,913 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:38,913 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:38,913 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,913 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:38,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:38,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:38,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,914 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,915 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:38,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-24 12:16:38,916 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@74f36261[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:38,916 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-24 12:16:38,916 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,917 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,917 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,917 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,917 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:38,917 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:38,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,917 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:38,917 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:38,917 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:38,917 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,917 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,917 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,917 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:38,917 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:38,917 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:38,917 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:38,917 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:38,922 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:38,922 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:38,922 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,922 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:38,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:38,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:38,923 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:38,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:38,923 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:38,923 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:38,923 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:38,923 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:38,923 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:38,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:38,923 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:38,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:38,926 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:16:38,926 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-24 12:16:38,926 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:38,926 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:38,926 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,926 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:38,927 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,927 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,927 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,928 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,928 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,927 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,928 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:38,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:38,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:38,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,928 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,928 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,928 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,929 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:38,929 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:38,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,931 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,931 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,932 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:38,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-24 12:16:38,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:38,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,936 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:38,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:38,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:38,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:38,937 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:38,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,937 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:38,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:38,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:38,937 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,937 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:38,937 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,937 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:38,937 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:38,937 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:38,937 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,037 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:39,038 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:39,049 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:39,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:39,050 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,050 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-24 12:16:39,050 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:39,050 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-24 12:16:39,050 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-24 12:16:39,051 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,051 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,052 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,052 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,052 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,052 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,052 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,053 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,053 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:39,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,053 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,053 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,053 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:39,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:39,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,053 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,053 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:39,053 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,054 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,054 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:39,054 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,054 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:39,054 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:39,054 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:39,054 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:39,054 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:39,054 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. started... 2023-07-24 12:16:39,054 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:39,055 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff6186d783435ac95747fe76928d7a76 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:39,056 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,056 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,056 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,056 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:39,057 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,057 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:39,064 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf1/2c1e4a87b954439b99162cc5fba9f013 2023-07-24 12:16:39,075 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf2/efadf79b50904464a9643f63d493f32c 2023-07-24 12:16:39,078 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf1/2c1e4a87b954439b99162cc5fba9f013 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/2c1e4a87b954439b99162cc5fba9f013 2023-07-24 12:16:39,081 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/2c1e4a87b954439b99162cc5fba9f013, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:16:39,081 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf2/efadf79b50904464a9643f63d493f32c as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/efadf79b50904464a9643f63d493f32c 2023-07-24 12:16:39,084 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/efadf79b50904464a9643f63d493f32c, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:16:39,085 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff6186d783435ac95747fe76928d7a76 in 30ms, sequenceid=45, compaction requested=true 2023-07-24 12:16:39,085 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff6186d783435ac95747fe76928d7a76: 2023-07-24 12:16:39,085 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:39,085 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:39,085 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:39,085 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,085 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:39,085 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:39,088 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,088 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:39,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:39,088 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,088 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:39,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:39,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:39,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:39,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-24 12:16:39,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1fa1d6c2[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:39,090 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-24 12:16:39,090 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,091 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,091 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,091 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,092 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,092 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,092 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:39,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,092 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:39,092 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:39,092 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:39,092 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:39,092 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,092 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:39,092 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:39,092 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:39,092 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:39,092 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:39,093 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:39,093 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:39,093 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:39,093 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:39,093 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:39,093 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,093 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:39,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:39,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:39,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:39,094 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:39,094 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:39,094 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:39,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:39,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,094 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:39,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:39,096 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:16:39,096 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-24 12:16:39,097 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:39,097 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:39,097 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,097 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:39,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:39,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:39,101 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,101 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:39,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,102 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:39,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,102 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:39,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:39,102 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:39,102 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:39,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:39,103 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:39,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:39,104 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,104 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,104 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,105 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,105 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,109 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,109 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,109 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,110 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,109 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,110 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:39,110 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,110 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,110 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:39,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-24 12:16:39,110 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:39,110 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:39,110 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:39,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,110 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:39,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:39,110 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:39,110 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,110 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,110 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:39,111 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,111 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:39,111 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:39,111 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,111 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,111 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,111 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:39,111 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:39,113 ERROR [BP-51740533-172.31.2.20-1690200889613 heartbeating to localhost/127.0.0.1:33821] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-51740533-172.31.2.20-1690200889613 (Datanode Uuid 295fb89a-ce04-4534-9158-8d1eb7f1a7e1) service to localhost/127.0.0.1:33821 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-24 12:16:39,211 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:39,212 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:39,223 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:39,224 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:39,225 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,225 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-24 12:16:39,225 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:39,225 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-24 12:16:39,225 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-24 12:16:39,225 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,225 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,226 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,226 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,226 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,226 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,226 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,226 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,227 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,227 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:39,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,227 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,227 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,227 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:39,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:39,228 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:39,228 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:39,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:39,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,228 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,228 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:39,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,228 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:39,228 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,228 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:39,228 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:39,229 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:39,228 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:39,229 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:39,229 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:39,229 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:39,229 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:39,229 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:39,229 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:39,229 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:39,229 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:39,229 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:39,229 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:39,229 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:39,229 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. started... 2023-07-24 12:16:39,229 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff6186d783435ac95747fe76928d7a76 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:39,230 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,230 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,231 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,231 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,231 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:39,231 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:39,241 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf1/4d6fd562eed3408681f8c09204c47483 2023-07-24 12:16:39,250 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf2/b2b65d66e96c473dbde2a787a862feda 2023-07-24 12:16:39,254 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf1/4d6fd562eed3408681f8c09204c47483 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/4d6fd562eed3408681f8c09204c47483 2023-07-24 12:16:39,257 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/4d6fd562eed3408681f8c09204c47483, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:16:39,258 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf2/b2b65d66e96c473dbde2a787a862feda as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/b2b65d66e96c473dbde2a787a862feda 2023-07-24 12:16:39,260 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/b2b65d66e96c473dbde2a787a862feda, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:16:39,261 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff6186d783435ac95747fe76928d7a76 in 32ms, sequenceid=58, compaction requested=true 2023-07-24 12:16:39,261 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff6186d783435ac95747fe76928d7a76: 2023-07-24 12:16:39,261 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:39,261 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:39,261 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:39,261 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,261 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:39,261 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:39,266 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,266 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:39,266 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:39,266 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,267 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:39,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:39,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:39,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:39,269 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-24 12:16:39,269 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@4de9bd11[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:39,269 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-24 12:16:39,269 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,270 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,270 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,270 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,270 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,270 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:39,270 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,270 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:39,270 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:39,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,270 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:39,270 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:39,270 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:39,270 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:39,270 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:39,270 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:39,270 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:39,271 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:39,271 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:39,271 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:39,272 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,272 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:39,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:39,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:39,272 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:39,272 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:39,272 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:39,272 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:39,272 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:39,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:39,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:39,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:39,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,274 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,275 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:39,275 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:16:39,275 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-24 12:16:39,275 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:39,275 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:39,275 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,276 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:39,283 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,283 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,283 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,283 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:39,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:39,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:39,283 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,283 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:39,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,283 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,283 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:39,283 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,283 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:39,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:39,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:39,284 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:39,284 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:39,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:39,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:39,284 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:39,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,284 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,285 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,286 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:39,286 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,286 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,286 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,287 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,289 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,289 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,289 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,289 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,289 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,289 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:39,289 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:39,289 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,289 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-24 12:16:39,291 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:39,289 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,289 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,289 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,291 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:39,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,291 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:39,291 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:39,291 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,291 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:39,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:39,291 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,291 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:39,292 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:39,292 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:39,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:39,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:39,292 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,292 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:39,292 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:39,292 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,292 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,392 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:39,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:39,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:39,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegionEmptyDaughter was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:39,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,404 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegionEmptyDaughter' 2023-07-24 12:16:39,404 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:39,404 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' 2023-07-24 12:16:39,404 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegionEmptyDaughter', kicking off acquire phase on members. 2023-07-24 12:16:39,405 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,405 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,406 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,406 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,406 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,406 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:39,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,406 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:39,406 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:39,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,406 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:39,406 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:39,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,407 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:39,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:39,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:39,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,407 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,407 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,408 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,407 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,408 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:39,408 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:39,408 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,408 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,408 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:39,408 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:39,408 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:39,408 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:39,408 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:39,408 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:39,408 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:39,408 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:39,408 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegionEmptyDaughter is still present, removing 2023-07-24 12:16:39,409 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,409 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:39,409 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegionEmptyDaughter' with timeout 60000ms 2023-07-24 12:16:39,409 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:39,409 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:39,409 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:39,409 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegionEmptyDaughter' starting 'acquire' stage 2023-07-24 12:16:39,409 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. started... 2023-07-24 12:16:39,409 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:39,409 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:39,409 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing ff6186d783435ac95747fe76928d7a76 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:39,410 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,412 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,412 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,412 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:39,412 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:39,412 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:39,818 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf1/4e8a2de75aa54cd8ac7ac1071b095f87 2023-07-24 12:16:40,230 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf2/58361385a4ed42098e5c4e6a544681cc 2023-07-24 12:16:40,234 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf1/4e8a2de75aa54cd8ac7ac1071b095f87 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/4e8a2de75aa54cd8ac7ac1071b095f87 2023-07-24 12:16:40,237 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/4e8a2de75aa54cd8ac7ac1071b095f87, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:16:40,238 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf2/58361385a4ed42098e5c4e6a544681cc as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/58361385a4ed42098e5c4e6a544681cc 2023-07-24 12:16:40,241 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/58361385a4ed42098e5c4e6a544681cc, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:16:40,241 INFO [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for ff6186d783435ac95747fe76928d7a76 in 832ms, sequenceid=71, compaction requested=true 2023-07-24 12:16:40,242 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for ff6186d783435ac95747fe76928d7a76: 2023-07-24 12:16:40,242 DEBUG [rs(jenkins-hbase6.apache.org,34575,1690200894532)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:40,242 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:40,242 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:40,242 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,242 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegionEmptyDaughter' locally acquired 2023-07-24 12:16:40,242 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegionEmptyDaughter) in zk 2023-07-24 12:16:40,243 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,243 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:40,243 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:40,244 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,244 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegionEmptyDaughter' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:40,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:40,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:40,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,244 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:40,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:40,245 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:40,245 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure 'testSplitTableRegionEmptyDaughter' on coordinator 2023-07-24 12:16:40,245 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@7796cfe2[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:40,245 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegionEmptyDaughter' starting 'in-barrier' execution. 2023-07-24 12:16:40,246 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,249 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,249 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,249 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,250 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,249 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,250 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:40,250 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,250 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:40,250 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegionEmptyDaughter' received 'reached' from coordinator. 2023-07-24 12:16:40,250 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:40,250 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,250 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:40,250 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:40,250 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:40,250 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegionEmptyDaughter' locally completed 2023-07-24 12:16:40,250 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:40,250 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegionEmptyDaughter' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:40,252 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:40,252 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,252 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:40,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:40,252 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:40,252 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:40,253 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:40,253 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegionEmptyDaughter' has notified controller of completion 2023-07-24 12:16:40,253 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:40,253 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:40,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:40,253 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:40,253 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegionEmptyDaughter' completed. 2023-07-24 12:16:40,253 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:40,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:40,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:40,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:40,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:40,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:40,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegionEmptyDaughter' member 'jenkins-hbase6.apache.org,34575,1690200894532': 2023-07-24 12:16:40,256 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' released barrier for procedure'testSplitTableRegionEmptyDaughter', counting down latch. Waiting for 0 more 2023-07-24 12:16:40,256 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegionEmptyDaughter' execution completed 2023-07-24 12:16:40,256 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:40,256 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:40,256 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,256 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionEmptyDaughterincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:40,257 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,257 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,257 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,257 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,258 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:40,257 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:40,257 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:40,257 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:40,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:40,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:40,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:40,258 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:40,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:40,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:40,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:40,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:40,258 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:40,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,259 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:40,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:40,259 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:40,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:40,260 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,260 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:40,261 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:40,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,261 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:40,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:40,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:40,263 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:40,263 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:40,263 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:40,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:40,263 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:40,263 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegionEmptyDaughter' 2023-07-24 12:16:40,263 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:40,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:40,263 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:40,263 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:40,263 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:40,263 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:40,264 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegionEmptyDaughter'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:40,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:40,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:40,264 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:40,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:40,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:40,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:40,263 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:40,264 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:40,264 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:40,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:40,264 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:40,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:40,264 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,264 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:40,264 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,264 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:40,264 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,264 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,265 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegionEmptyDaughter 2023-07-24 12:16:40,364 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:40,365 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:40,371 DEBUG [Listener at localhost/37279] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,372 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=ff6186d783435ac95747fe76928d7a76, daughterA=e874484af740f9942cb9359ddd22ed10, daughterB=8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:40,373 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=ff6186d783435ac95747fe76928d7a76, daughterA=e874484af740f9942cb9359ddd22ed10, daughterB=8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:40,373 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=ff6186d783435ac95747fe76928d7a76, daughterA=e874484af740f9942cb9359ddd22ed10, daughterB=8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:40,373 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=128, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=ff6186d783435ac95747fe76928d7a76, daughterA=e874484af740f9942cb9359ddd22ed10, daughterB=8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:40,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=ff6186d783435ac95747fe76928d7a76, UNASSIGN}] 2023-07-24 12:16:40,375 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=129, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=ff6186d783435ac95747fe76928d7a76, UNASSIGN 2023-07-24 12:16:40,376 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=ff6186d783435ac95747fe76928d7a76, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:40,376 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690201000376"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690201000376"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690201000376"}]},"ts":"1690201000376"} 2023-07-24 12:16:40,377 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=130, ppid=129, state=RUNNABLE; CloseRegionProcedure ff6186d783435ac95747fe76928d7a76, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:40,529 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing ff6186d783435ac95747fe76928d7a76, disabling compactions & flushes 2023-07-24 12:16:40,529 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:40,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:40,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. after waiting 0 ms 2023-07-24 12:16:40,529 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:40,529 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2745): Flushing ff6186d783435ac95747fe76928d7a76 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-24 12:16:40,538 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf1/bf8e165ca49f4e5c9f15fc298e7f39ec 2023-07-24 12:16:40,548 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf2/9642abacff2e478f8d68f8edfad7d2a8 2023-07-24 12:16:40,551 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf1/bf8e165ca49f4e5c9f15fc298e7f39ec as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/bf8e165ca49f4e5c9f15fc298e7f39ec 2023-07-24 12:16:40,554 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/bf8e165ca49f4e5c9f15fc298e7f39ec, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 12:16:40,554 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.tmp/cf2/9642abacff2e478f8d68f8edfad7d2a8 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/9642abacff2e478f8d68f8edfad7d2a8 2023-07-24 12:16:40,557 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/9642abacff2e478f8d68f8edfad7d2a8, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 12:16:40,558 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for ff6186d783435ac95747fe76928d7a76 in 29ms, sequenceid=82, compaction requested=true 2023-07-24 12:16:40,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-24 12:16:40,562 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:16:40,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. 2023-07-24 12:16:40,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for ff6186d783435ac95747fe76928d7a76: 2023-07-24 12:16:40,565 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,565 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=129 updating hbase:meta row=ff6186d783435ac95747fe76928d7a76, regionState=CLOSED 2023-07-24 12:16:40,565 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690201000565"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690201000565"}]},"ts":"1690201000565"} 2023-07-24 12:16:40,567 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=130, resume processing ppid=129 2023-07-24 12:16:40,568 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=130, ppid=129, state=SUCCESS; CloseRegionProcedure ff6186d783435ac95747fe76928d7a76, server=jenkins-hbase6.apache.org,34575,1690200894532 in 189 msec 2023-07-24 12:16:40,569 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=129, resume processing ppid=128 2023-07-24 12:16:40,569 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=129, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=ff6186d783435ac95747fe76928d7a76, UNASSIGN in 193 msec 2023-07-24 12:16:40,578 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=128 splitting 14 storefiles, region=ff6186d783435ac95747fe76928d7a76, threads=14 2023-07-24 12:16:40,578 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/58361385a4ed42098e5c4e6a544681cc for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,578 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/7e6c82f848bc414ca58e79844e14cf13 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,578 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/7f091c2de0c64121852ad00f8a4c34f7 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,578 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/9642abacff2e478f8d68f8edfad7d2a8 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,579 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/b2b65d66e96c473dbde2a787a862feda for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,579 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/efadf79b50904464a9643f63d493f32c for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,579 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/f354c1248fe94d4e95b7deb9a135f78b for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,579 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/2c1e4a87b954439b99162cc5fba9f013 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,579 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/4d6fd562eed3408681f8c09204c47483 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,579 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/4e8a2de75aa54cd8ac7ac1071b095f87 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,579 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/763fd36339d04267834a48b189e7c242 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,579 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/70cc4e48847f4ac88d8c9565e1ae6ee6 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,579 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/bf8e165ca49f4e5c9f15fc298e7f39ec for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,580 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=128 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/ee727bc1f2df45f0b1b6651003035058 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,585 DEBUG [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/bf8e165ca49f4e5c9f15fc298e7f39ec, top=false 2023-07-24 12:16:40,590 DEBUG [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/7e6c82f848bc414ca58e79844e14cf13, top=false 2023-07-24 12:16:40,590 DEBUG [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/763fd36339d04267834a48b189e7c242, top=false 2023-07-24 12:16:40,598 DEBUG [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/58361385a4ed42098e5c4e6a544681cc, top=false 2023-07-24 12:16:40,599 DEBUG [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/b2b65d66e96c473dbde2a787a862feda, top=false 2023-07-24 12:16:40,598 DEBUG [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/2c1e4a87b954439b99162cc5fba9f013, top=false 2023-07-24 12:16:40,600 DEBUG [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/7f091c2de0c64121852ad00f8a4c34f7, top=false 2023-07-24 12:16:40,600 DEBUG [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/4e8a2de75aa54cd8ac7ac1071b095f87, top=false 2023-07-24 12:16:40,600 DEBUG [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/efadf79b50904464a9643f63d493f32c, top=false 2023-07-24 12:16:40,598 DEBUG [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/f354c1248fe94d4e95b7deb9a135f78b, top=false 2023-07-24 12:16:40,600 DEBUG [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/70cc4e48847f4ac88d8c9565e1ae6ee6, top=false 2023-07-24 12:16:40,601 DEBUG [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/ee727bc1f2df45f0b1b6651003035058, top=false 2023-07-24 12:16:40,599 DEBUG [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/9642abacff2e478f8d68f8edfad7d2a8, top=false 2023-07-24 12:16:40,603 DEBUG [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(713): Will create HFileLink file for hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/4d6fd562eed3408681f8c09204c47483, top=false 2023-07-24 12:16:40,619 INFO [StoreFileSplitter-pool-11] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.splits/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-763fd36339d04267834a48b189e7c242 for child: e874484af740f9942cb9359ddd22ed10, parent: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,620 INFO [StoreFileSplitter-pool-12] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.splits/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-bf8e165ca49f4e5c9f15fc298e7f39ec for child: e874484af740f9942cb9359ddd22ed10, parent: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,624 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/763fd36339d04267834a48b189e7c242 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,625 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/bf8e165ca49f4e5c9f15fc298e7f39ec for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,630 INFO [StoreFileSplitter-pool-1] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.splits/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-7e6c82f848bc414ca58e79844e14cf13 for child: e874484af740f9942cb9359ddd22ed10, parent: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,633 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/7e6c82f848bc414ca58e79844e14cf13 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,636 INFO [StoreFileSplitter-pool-0] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.splits/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-58361385a4ed42098e5c4e6a544681cc for child: e874484af740f9942cb9359ddd22ed10, parent: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,637 INFO [StoreFileSplitter-pool-4] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.splits/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-b2b65d66e96c473dbde2a787a862feda for child: e874484af740f9942cb9359ddd22ed10, parent: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,639 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/58361385a4ed42098e5c4e6a544681cc for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,640 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/b2b65d66e96c473dbde2a787a862feda for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,643 INFO [StoreFileSplitter-pool-7] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.splits/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-2c1e4a87b954439b99162cc5fba9f013 for child: e874484af740f9942cb9359ddd22ed10, parent: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,643 INFO [StoreFileSplitter-pool-9] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.splits/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-4e8a2de75aa54cd8ac7ac1071b095f87 for child: e874484af740f9942cb9359ddd22ed10, parent: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,646 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/4e8a2de75aa54cd8ac7ac1071b095f87 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,646 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/2c1e4a87b954439b99162cc5fba9f013 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,647 INFO [StoreFileSplitter-pool-3] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.splits/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-9642abacff2e478f8d68f8edfad7d2a8 for child: e874484af740f9942cb9359ddd22ed10, parent: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,648 INFO [StoreFileSplitter-pool-13] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.splits/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-ee727bc1f2df45f0b1b6651003035058 for child: e874484af740f9942cb9359ddd22ed10, parent: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,650 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/9642abacff2e478f8d68f8edfad7d2a8 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,652 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/ee727bc1f2df45f0b1b6651003035058 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,655 INFO [StoreFileSplitter-pool-2] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.splits/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-7f091c2de0c64121852ad00f8a4c34f7 for child: e874484af740f9942cb9359ddd22ed10, parent: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,655 INFO [StoreFileSplitter-pool-8] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.splits/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-4d6fd562eed3408681f8c09204c47483 for child: e874484af740f9942cb9359ddd22ed10, parent: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,655 INFO [StoreFileSplitter-pool-10] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.splits/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-70cc4e48847f4ac88d8c9565e1ae6ee6 for child: e874484af740f9942cb9359ddd22ed10, parent: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,655 INFO [StoreFileSplitter-pool-6] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.splits/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-f354c1248fe94d4e95b7deb9a135f78b for child: e874484af740f9942cb9359ddd22ed10, parent: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,655 INFO [StoreFileSplitter-pool-5] regionserver.HRegionFileSystem(742): Created linkFile:hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/.splits/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-efadf79b50904464a9643f63d493f32c for child: e874484af740f9942cb9359ddd22ed10, parent: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,659 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/efadf79b50904464a9643f63d493f32c for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,659 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/7f091c2de0c64121852ad00f8a4c34f7 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,659 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/f354c1248fe94d4e95b7deb9a135f78b for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,661 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/4d6fd562eed3408681f8c09204c47483 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,666 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=128 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/70cc4e48847f4ac88d8c9565e1ae6ee6 for region: ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:40,666 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=128 split storefiles for region ff6186d783435ac95747fe76928d7a76 Daughter A: 14 storefiles, Daughter B: 0 storefiles. 2023-07-24 12:16:41,097 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 12:16:41,099 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 12:16:41,100 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76.","families":{"info":[{"qualifier":"regioninfo","vlen":67,"tag":[],"timestamp":"1690201001100"},{"qualifier":"splitA","vlen":69,"tag":[],"timestamp":"1690201001100"},{"qualifier":"splitB","vlen":69,"tag":[],"timestamp":"1690201001100"}]},"ts":"1690201001100"} 2023-07-24 12:16:41,100 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690201001100"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690201001100"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690201001100"}]},"ts":"1690201001100"} 2023-07-24 12:16:41,100 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690201001100"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690201001100"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690201001100"}]},"ts":"1690201001100"} 2023-07-24 12:16:41,103 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e874484af740f9942cb9359ddd22ed10, ASSIGN}, {pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8691b3a4f0e752b460f46e505ee0aeb4, ASSIGN}] 2023-07-24 12:16:41,103 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8691b3a4f0e752b460f46e505ee0aeb4, ASSIGN 2023-07-24 12:16:41,104 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=132, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8691b3a4f0e752b460f46e505ee0aeb4, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase6.apache.org,34575,1690200894532; forceNewPlan=false, retain=false 2023-07-24 12:16:41,105 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e874484af740f9942cb9359ddd22ed10, ASSIGN 2023-07-24 12:16:41,105 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=131, ppid=128, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e874484af740f9942cb9359ddd22ed10, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase6.apache.org,34575,1690200894532; forceNewPlan=false, retain=false 2023-07-24 12:16:41,254 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-24 12:16:41,256 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=e874484af740f9942cb9359ddd22ed10, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:41,256 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690201001256"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690201001256"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690201001256"}]},"ts":"1690201001256"} 2023-07-24 12:16:41,257 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=133, ppid=131, state=RUNNABLE; OpenRegionProcedure e874484af740f9942cb9359ddd22ed10, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:41,259 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=8691b3a4f0e752b460f46e505ee0aeb4, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:41,259 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690201001259"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690201001259"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690201001259"}]},"ts":"1690201001259"} 2023-07-24 12:16:41,260 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=134, ppid=132, state=RUNNABLE; OpenRegionProcedure 8691b3a4f0e752b460f46e505ee0aeb4, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:41,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4. 2023-07-24 12:16:41,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8691b3a4f0e752b460f46e505ee0aeb4, NAME => 'testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4.', STARTKEY => '71', ENDKEY => ''} 2023-07-24 12:16:41,411 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:16:41,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter 8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:41,411 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:16:41,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:41,412 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:41,413 INFO [StoreOpener-8691b3a4f0e752b460f46e505ee0aeb4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:41,414 DEBUG [StoreOpener-8691b3a4f0e752b460f46e505ee0aeb4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4/cf1 2023-07-24 12:16:41,414 DEBUG [StoreOpener-8691b3a4f0e752b460f46e505ee0aeb4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4/cf1 2023-07-24 12:16:41,414 INFO [StoreOpener-8691b3a4f0e752b460f46e505ee0aeb4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8691b3a4f0e752b460f46e505ee0aeb4 columnFamilyName cf1 2023-07-24 12:16:41,415 INFO [StoreOpener-8691b3a4f0e752b460f46e505ee0aeb4-1] regionserver.HStore(310): Store=8691b3a4f0e752b460f46e505ee0aeb4/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:41,415 INFO [StoreOpener-8691b3a4f0e752b460f46e505ee0aeb4-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:41,416 DEBUG [StoreOpener-8691b3a4f0e752b460f46e505ee0aeb4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4/cf2 2023-07-24 12:16:41,416 DEBUG [StoreOpener-8691b3a4f0e752b460f46e505ee0aeb4-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4/cf2 2023-07-24 12:16:41,416 INFO [StoreOpener-8691b3a4f0e752b460f46e505ee0aeb4-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8691b3a4f0e752b460f46e505ee0aeb4 columnFamilyName cf2 2023-07-24 12:16:41,417 INFO [StoreOpener-8691b3a4f0e752b460f46e505ee0aeb4-1] regionserver.HStore(310): Store=8691b3a4f0e752b460f46e505ee0aeb4/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:41,417 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:41,418 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:41,419 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:16:41,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:41,420 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 8691b3a4f0e752b460f46e505ee0aeb4; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=72229929, jitterRate=0.07630981504917145}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:16:41,420 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 8691b3a4f0e752b460f46e505ee0aeb4: 2023-07-24 12:16:41,421 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4., pid=134, masterSystemTime=1690201001408 2023-07-24 12:16:41,422 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4. 2023-07-24 12:16:41,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4. 2023-07-24 12:16:41,422 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10. 2023-07-24 12:16:41,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => e874484af740f9942cb9359ddd22ed10, NAME => 'testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.', STARTKEY => '', ENDKEY => '71'} 2023-07-24 12:16:41,423 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=132 updating hbase:meta row=8691b3a4f0e752b460f46e505ee0aeb4, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:41,423 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690201001423"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690201001423"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690201001423"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690201001423"}]},"ts":"1690201001423"} 2023-07-24 12:16:41,423 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:16:41,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegionEmptyDaughter e874484af740f9942cb9359ddd22ed10 2023-07-24 12:16:41,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:16:41,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for e874484af740f9942cb9359ddd22ed10 2023-07-24 12:16:41,423 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for e874484af740f9942cb9359ddd22ed10 2023-07-24 12:16:41,424 INFO [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region e874484af740f9942cb9359ddd22ed10 2023-07-24 12:16:41,425 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1 2023-07-24 12:16:41,426 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1 2023-07-24 12:16:41,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=134, resume processing ppid=132 2023-07-24 12:16:41,426 INFO [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e874484af740f9942cb9359ddd22ed10 columnFamilyName cf1 2023-07-24 12:16:41,426 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=134, ppid=132, state=SUCCESS; OpenRegionProcedure 8691b3a4f0e752b460f46e505ee0aeb4, server=jenkins-hbase6.apache.org,34575,1690200894532 in 164 msec 2023-07-24 12:16:41,427 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=132, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8691b3a4f0e752b460f46e505ee0aeb4, ASSIGN in 323 msec 2023-07-24 12:16:41,432 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-2c1e4a87b954439b99162cc5fba9f013 2023-07-24 12:16:41,434 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-4d6fd562eed3408681f8c09204c47483 2023-07-24 12:16:41,437 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-4e8a2de75aa54cd8ac7ac1071b095f87 2023-07-24 12:16:41,439 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-70cc4e48847f4ac88d8c9565e1ae6ee6 2023-07-24 12:16:41,442 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-763fd36339d04267834a48b189e7c242 2023-07-24 12:16:41,444 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-bf8e165ca49f4e5c9f15fc298e7f39ec 2023-07-24 12:16:41,446 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-ee727bc1f2df45f0b1b6651003035058 2023-07-24 12:16:41,447 INFO [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(310): Store=e874484af740f9942cb9359ddd22ed10/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:41,447 INFO [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region e874484af740f9942cb9359ddd22ed10 2023-07-24 12:16:41,448 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2 2023-07-24 12:16:41,448 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2 2023-07-24 12:16:41,448 INFO [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region e874484af740f9942cb9359ddd22ed10 columnFamilyName cf2 2023-07-24 12:16:41,453 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-58361385a4ed42098e5c4e6a544681cc 2023-07-24 12:16:41,456 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-7e6c82f848bc414ca58e79844e14cf13 2023-07-24 12:16:41,458 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-7f091c2de0c64121852ad00f8a4c34f7 2023-07-24 12:16:41,460 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-9642abacff2e478f8d68f8edfad7d2a8 2023-07-24 12:16:41,463 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-b2b65d66e96c473dbde2a787a862feda 2023-07-24 12:16:41,465 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-efadf79b50904464a9643f63d493f32c 2023-07-24 12:16:41,468 DEBUG [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-f354c1248fe94d4e95b7deb9a135f78b 2023-07-24 12:16:41,468 INFO [StoreOpener-e874484af740f9942cb9359ddd22ed10-1] regionserver.HStore(310): Store=e874484af740f9942cb9359ddd22ed10/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:41,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10 2023-07-24 12:16:41,469 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10 2023-07-24 12:16:41,471 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegionEmptyDaughter descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:16:41,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for e874484af740f9942cb9359ddd22ed10 2023-07-24 12:16:41,472 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened e874484af740f9942cb9359ddd22ed10; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=65241059, jitterRate=-0.02783246338367462}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:16:41,472 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for e874484af740f9942cb9359ddd22ed10: 2023-07-24 12:16:41,473 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10., pid=133, masterSystemTime=1690201001408 2023-07-24 12:16:41,473 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:41,474 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 12:16:41,474 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:16:41,474 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:16:41,474 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-24 12:16:41,474 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.ExploringCompactionPolicy(116): Exploring compaction algorithm has selected 7 files of size 35728 starting at candidate #0 after considering 15 permutations with 15 in ratio 2023-07-24 12:16:41,474 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1912): e874484af740f9942cb9359ddd22ed10/cf1 is initiating minor compaction (all files) 2023-07-24 12:16:41,474 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1912): e874484af740f9942cb9359ddd22ed10/cf2 is initiating minor compaction (all files) 2023-07-24 12:16:41,474 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of e874484af740f9942cb9359ddd22ed10/cf1 in testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10. 2023-07-24 12:16:41,474 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegion(2259): Starting compaction of e874484af740f9942cb9359ddd22ed10/cf2 in testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10. 2023-07-24 12:16:41,474 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-ee727bc1f2df45f0b1b6651003035058, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-70cc4e48847f4ac88d8c9565e1ae6ee6, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-763fd36339d04267834a48b189e7c242, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-2c1e4a87b954439b99162cc5fba9f013, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-4d6fd562eed3408681f8c09204c47483, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-4e8a2de75aa54cd8ac7ac1071b095f87, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-bf8e165ca49f4e5c9f15fc298e7f39ec] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/.tmp, totalSize=34.9 K 2023-07-24 12:16:41,474 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-f354c1248fe94d4e95b7deb9a135f78b, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-7f091c2de0c64121852ad00f8a4c34f7, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-7e6c82f848bc414ca58e79844e14cf13, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-efadf79b50904464a9643f63d493f32c, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-b2b65d66e96c473dbde2a787a862feda, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-58361385a4ed42098e5c4e6a544681cc, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-9642abacff2e478f8d68f8edfad7d2a8] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/.tmp, totalSize=34.9 K 2023-07-24 12:16:41,475 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-ee727bc1f2df45f0b1b6651003035058, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690200998515 2023-07-24 12:16:41,475 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-f354c1248fe94d4e95b7deb9a135f78b, keycount=2, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690200998515 2023-07-24 12:16:41,475 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10. 2023-07-24 12:16:41,475 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10. 2023-07-24 12:16:41,475 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-70cc4e48847f4ac88d8c9565e1ae6ee6, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690200998681 2023-07-24 12:16:41,476 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=131 updating hbase:meta row=e874484af740f9942cb9359ddd22ed10, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:41,476 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-7f091c2de0c64121852ad00f8a4c34f7, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690200998681 2023-07-24 12:16:41,476 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-763fd36339d04267834a48b189e7c242, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690200998853 2023-07-24 12:16:41,476 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690201001476"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690201001476"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690201001476"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690201001476"}]},"ts":"1690201001476"} 2023-07-24 12:16:41,476 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-7e6c82f848bc414ca58e79844e14cf13, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690200998853 2023-07-24 12:16:41,476 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-2c1e4a87b954439b99162cc5fba9f013, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690200999038 2023-07-24 12:16:41,477 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-efadf79b50904464a9643f63d493f32c, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690200999038 2023-07-24 12:16:41,477 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-4d6fd562eed3408681f8c09204c47483, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690200999213 2023-07-24 12:16:41,477 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-b2b65d66e96c473dbde2a787a862feda, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690200999213 2023-07-24 12:16:41,477 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-4e8a2de75aa54cd8ac7ac1071b095f87, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690200999394 2023-07-24 12:16:41,478 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-58361385a4ed42098e5c4e6a544681cc, keycount=10, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690200999394 2023-07-24 12:16:41,478 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-bf8e165ca49f4e5c9f15fc298e7f39ec, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690201000366 2023-07-24 12:16:41,478 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] compactions.Compactor(207): Compacting testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-9642abacff2e478f8d68f8edfad7d2a8, keycount=8, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690201000366 2023-07-24 12:16:41,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=133, resume processing ppid=131 2023-07-24 12:16:41,479 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=133, ppid=131, state=SUCCESS; OpenRegionProcedure e874484af740f9942cb9359ddd22ed10, server=jenkins-hbase6.apache.org,34575,1690200894532 in 220 msec 2023-07-24 12:16:41,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=131, resume processing ppid=128 2023-07-24 12:16:41,481 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=131, ppid=128, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e874484af740f9942cb9359ddd22ed10, ASSIGN in 376 msec 2023-07-24 12:16:41,483 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=128, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegionEmptyDaughter, parent=ff6186d783435ac95747fe76928d7a76, daughterA=e874484af740f9942cb9359ddd22ed10, daughterB=8691b3a4f0e752b460f46e505ee0aeb4 in 1.1100 sec 2023-07-24 12:16:41,497 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] throttle.PressureAwareThroughputController(145): e874484af740f9942cb9359ddd22ed10#cf2#compaction#126 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:16:41,498 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] throttle.PressureAwareThroughputController(145): e874484af740f9942cb9359ddd22ed10#cf1#compaction#127 average throughput is 1.72 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:16:41,624 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=34575] ipc.CallRunner(144): callId: 987 service: ClientService methodName: Scan size: 157 connection: 172.31.2.20:46684 deadline: 1690201061623, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76. is not online on jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:41,749 DEBUG [Listener at localhost/37279] client.ClientScanner(240): Advancing internal scanner to startKey at '71', inclusive 2023-07-24 12:16:41,752 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:16:41,753 INFO [Listener at localhost/37279] client.HBaseAdmin$15(890): Started disable of testSplitTableRegionEmptyDaughter 2023-07-24 12:16:41,753 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$11(2418): Client=jenkins//172.31.2.20 disable testSplitTableRegionEmptyDaughter 2023-07-24 12:16:41,754 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=135, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 12:16:41,756 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-07-24 12:16:41,756 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690201001756"}]},"ts":"1690201001756"} 2023-07-24 12:16:41,757 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLING in hbase:meta 2023-07-24 12:16:41,759 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegionEmptyDaughter to state=DISABLING 2023-07-24 12:16:41,760 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e874484af740f9942cb9359ddd22ed10, UNASSIGN}, {pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8691b3a4f0e752b460f46e505ee0aeb4, UNASSIGN}] 2023-07-24 12:16:41,761 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=137, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8691b3a4f0e752b460f46e505ee0aeb4, UNASSIGN 2023-07-24 12:16:41,761 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=8691b3a4f0e752b460f46e505ee0aeb4, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:41,761 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690201001761"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690201001761"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690201001761"}]},"ts":"1690201001761"} 2023-07-24 12:16:41,762 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=138, ppid=137, state=RUNNABLE; CloseRegionProcedure 8691b3a4f0e752b460f46e505ee0aeb4, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:41,764 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=136, ppid=135, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e874484af740f9942cb9359ddd22ed10, UNASSIGN 2023-07-24 12:16:41,764 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=e874484af740f9942cb9359ddd22ed10, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:41,765 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690201001764"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690201001764"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690201001764"}]},"ts":"1690201001764"} 2023-07-24 12:16:41,765 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=139, ppid=136, state=RUNNABLE; CloseRegionProcedure e874484af740f9942cb9359ddd22ed10, server=jenkins-hbase6.apache.org,34575,1690200894532}] 2023-07-24 12:16:41,857 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-07-24 12:16:41,911 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/.tmp/cf2/a567042c8bdb4ade82592de43d4dc41d as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/a567042c8bdb4ade82592de43d4dc41d 2023-07-24 12:16:41,914 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:41,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 8691b3a4f0e752b460f46e505ee0aeb4, disabling compactions & flushes 2023-07-24 12:16:41,915 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4. 2023-07-24 12:16:41,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4. 2023-07-24 12:16:41,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4. after waiting 0 ms 2023-07-24 12:16:41,915 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4. 2023-07-24 12:16:41,916 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e874484af740f9942cb9359ddd22ed10/cf2 of e874484af740f9942cb9359ddd22ed10 into a567042c8bdb4ade82592de43d4dc41d(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:16:41,916 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for e874484af740f9942cb9359ddd22ed10: 2023-07-24 12:16:41,916 INFO [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10., storeName=e874484af740f9942cb9359ddd22ed10/cf2, priority=9, startTime=1690201001474; duration=0sec 2023-07-24 12:16:41,917 DEBUG [RS:1;jenkins-hbase6:34575-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:41,919 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/.tmp/cf1/699e15fa9f2c4ef39e5b5c51ec05e52e as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/699e15fa9f2c4ef39e5b5c51ec05e52e 2023-07-24 12:16:41,919 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4/recovered.edits/88.seqid, newMaxSeqId=88, maxSeqId=85 2023-07-24 12:16:41,920 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:16:41,920 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4. 2023-07-24 12:16:41,920 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 8691b3a4f0e752b460f46e505ee0aeb4: 2023-07-24 12:16:41,921 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:41,921 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close e874484af740f9942cb9359ddd22ed10 2023-07-24 12:16:41,921 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing e874484af740f9942cb9359ddd22ed10, disabling compactions & flushes 2023-07-24 12:16:41,921 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1858): waiting for 1 compactions to complete for region testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10. 2023-07-24 12:16:41,922 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=137 updating hbase:meta row=8691b3a4f0e752b460f46e505ee0aeb4, regionState=CLOSED 2023-07-24 12:16:41,922 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690201001922"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690201001922"}]},"ts":"1690201001922"} 2023-07-24 12:16:41,923 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in e874484af740f9942cb9359ddd22ed10/cf1 of e874484af740f9942cb9359ddd22ed10 into 699e15fa9f2c4ef39e5b5c51ec05e52e(size=7.3 K), total size for store is 7.3 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:16:41,923 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for e874484af740f9942cb9359ddd22ed10: 2023-07-24 12:16:41,923 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10. 2023-07-24 12:16:41,923 INFO [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10., storeName=e874484af740f9942cb9359ddd22ed10/cf1, priority=9, startTime=1690201001473; duration=0sec 2023-07-24 12:16:41,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10. 2023-07-24 12:16:41,923 DEBUG [RS:1;jenkins-hbase6:34575-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:41,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10. after waiting 0 ms 2023-07-24 12:16:41,923 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10. 2023-07-24 12:16:41,924 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-ee727bc1f2df45f0b1b6651003035058, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-70cc4e48847f4ac88d8c9565e1ae6ee6, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-763fd36339d04267834a48b189e7c242, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-2c1e4a87b954439b99162cc5fba9f013, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-4d6fd562eed3408681f8c09204c47483, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-4e8a2de75aa54cd8ac7ac1071b095f87, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-bf8e165ca49f4e5c9f15fc298e7f39ec] to archive 2023-07-24 12:16:41,924 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=138, resume processing ppid=137 2023-07-24 12:16:41,925 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:16:41,925 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=138, ppid=137, state=SUCCESS; CloseRegionProcedure 8691b3a4f0e752b460f46e505ee0aeb4, server=jenkins-hbase6.apache.org,34575,1690200894532 in 161 msec 2023-07-24 12:16:41,926 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=137, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=8691b3a4f0e752b460f46e505ee0aeb4, UNASSIGN in 165 msec 2023-07-24 12:16:41,926 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-ee727bc1f2df45f0b1b6651003035058 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-ee727bc1f2df45f0b1b6651003035058 2023-07-24 12:16:41,927 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-70cc4e48847f4ac88d8c9565e1ae6ee6 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-70cc4e48847f4ac88d8c9565e1ae6ee6 2023-07-24 12:16:41,928 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-763fd36339d04267834a48b189e7c242 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-763fd36339d04267834a48b189e7c242 2023-07-24 12:16:41,929 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-2c1e4a87b954439b99162cc5fba9f013 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-2c1e4a87b954439b99162cc5fba9f013 2023-07-24 12:16:41,930 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-4d6fd562eed3408681f8c09204c47483 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-4d6fd562eed3408681f8c09204c47483 2023-07-24 12:16:41,931 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-4e8a2de75aa54cd8ac7ac1071b095f87 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-4e8a2de75aa54cd8ac7ac1071b095f87 2023-07-24 12:16:41,932 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-bf8e165ca49f4e5c9f15fc298e7f39ec to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-bf8e165ca49f4e5c9f15fc298e7f39ec 2023-07-24 12:16:41,933 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-f354c1248fe94d4e95b7deb9a135f78b, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-7f091c2de0c64121852ad00f8a4c34f7, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-7e6c82f848bc414ca58e79844e14cf13, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-efadf79b50904464a9643f63d493f32c, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-b2b65d66e96c473dbde2a787a862feda, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-58361385a4ed42098e5c4e6a544681cc, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-9642abacff2e478f8d68f8edfad7d2a8] to archive 2023-07-24 12:16:41,934 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:16:41,935 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-f354c1248fe94d4e95b7deb9a135f78b to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-f354c1248fe94d4e95b7deb9a135f78b 2023-07-24 12:16:41,936 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-7f091c2de0c64121852ad00f8a4c34f7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-7f091c2de0c64121852ad00f8a4c34f7 2023-07-24 12:16:41,937 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-7e6c82f848bc414ca58e79844e14cf13 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-7e6c82f848bc414ca58e79844e14cf13 2023-07-24 12:16:41,938 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-efadf79b50904464a9643f63d493f32c to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-efadf79b50904464a9643f63d493f32c 2023-07-24 12:16:41,939 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-b2b65d66e96c473dbde2a787a862feda to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-b2b65d66e96c473dbde2a787a862feda 2023-07-24 12:16:41,939 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-58361385a4ed42098e5c4e6a544681cc to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-58361385a4ed42098e5c4e6a544681cc 2023-07-24 12:16:41,941 DEBUG [StoreCloser-testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-9642abacff2e478f8d68f8edfad7d2a8 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/testSplitTableRegionEmptyDaughter=ff6186d783435ac95747fe76928d7a76-9642abacff2e478f8d68f8edfad7d2a8 2023-07-24 12:16:41,943 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 12:16:41,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:16:41,944 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10. 2023-07-24 12:16:41,944 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for e874484af740f9942cb9359ddd22ed10: 2023-07-24 12:16:41,945 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed e874484af740f9942cb9359ddd22ed10 2023-07-24 12:16:41,945 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=136 updating hbase:meta row=e874484af740f9942cb9359ddd22ed10, regionState=CLOSED 2023-07-24 12:16:41,946 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.","families":{"info":[{"qualifier":"regioninfo","vlen":69,"tag":[],"timestamp":"1690201001945"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690201001945"}]},"ts":"1690201001945"} 2023-07-24 12:16:41,947 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=139, resume processing ppid=136 2023-07-24 12:16:41,948 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=139, ppid=136, state=SUCCESS; CloseRegionProcedure e874484af740f9942cb9359ddd22ed10, server=jenkins-hbase6.apache.org,34575,1690200894532 in 181 msec 2023-07-24 12:16:41,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=136, resume processing ppid=135 2023-07-24 12:16:41,949 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=136, ppid=135, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegionEmptyDaughter, region=e874484af740f9942cb9359ddd22ed10, UNASSIGN in 188 msec 2023-07-24 12:16:41,950 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690201001949"}]},"ts":"1690201001949"} 2023-07-24 12:16:41,950 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegionEmptyDaughter, state=DISABLED in hbase:meta 2023-07-24 12:16:41,952 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegionEmptyDaughter to state=DISABLED 2023-07-24 12:16:41,953 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=135, state=SUCCESS; DisableTableProcedure table=testSplitTableRegionEmptyDaughter in 198 msec 2023-07-24 12:16:42,057 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=135 2023-07-24 12:16:42,058 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 135 completed 2023-07-24 12:16:42,058 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$5(2228): Client=jenkins//172.31.2.20 delete testSplitTableRegionEmptyDaughter 2023-07-24 12:16:42,059 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 12:16:42,060 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=140, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 12:16:42,061 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=140, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 12:16:42,061 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-07-24 12:16:42,065 DEBUG [HFileArchiver-9] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:42,065 DEBUG [HFileArchiver-11] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:42,065 DEBUG [HFileArchiver-10] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10 2023-07-24 12:16:42,067 DEBUG [HFileArchiver-11] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4/recovered.edits] 2023-07-24 12:16:42,071 DEBUG [HFileArchiver-10] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/recovered.edits] 2023-07-24 12:16:42,072 DEBUG [HFileArchiver-9] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/recovered.edits] 2023-07-24 12:16:42,075 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/699e15fa9f2c4ef39e5b5c51ec05e52e to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf1/699e15fa9f2c4ef39e5b5c51ec05e52e 2023-07-24 12:16:42,075 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/2c1e4a87b954439b99162cc5fba9f013 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/2c1e4a87b954439b99162cc5fba9f013 2023-07-24 12:16:42,077 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/4d6fd562eed3408681f8c09204c47483 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/4d6fd562eed3408681f8c09204c47483 2023-07-24 12:16:42,077 DEBUG [HFileArchiver-11] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4/recovered.edits/88.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4/recovered.edits/88.seqid 2023-07-24 12:16:42,078 DEBUG [HFileArchiver-11] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/8691b3a4f0e752b460f46e505ee0aeb4 2023-07-24 12:16:42,078 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/a567042c8bdb4ade82592de43d4dc41d to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/cf2/a567042c8bdb4ade82592de43d4dc41d 2023-07-24 12:16:42,078 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/4e8a2de75aa54cd8ac7ac1071b095f87 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/4e8a2de75aa54cd8ac7ac1071b095f87 2023-07-24 12:16:42,080 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/70cc4e48847f4ac88d8c9565e1ae6ee6 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/70cc4e48847f4ac88d8c9565e1ae6ee6 2023-07-24 12:16:42,081 DEBUG [HFileArchiver-10] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/recovered.edits/90.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10/recovered.edits/90.seqid 2023-07-24 12:16:42,081 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/763fd36339d04267834a48b189e7c242 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/763fd36339d04267834a48b189e7c242 2023-07-24 12:16:42,081 DEBUG [HFileArchiver-10] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/e874484af740f9942cb9359ddd22ed10 2023-07-24 12:16:42,082 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/bf8e165ca49f4e5c9f15fc298e7f39ec to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/bf8e165ca49f4e5c9f15fc298e7f39ec 2023-07-24 12:16:42,083 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/ee727bc1f2df45f0b1b6651003035058 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf1/ee727bc1f2df45f0b1b6651003035058 2023-07-24 12:16:42,085 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/58361385a4ed42098e5c4e6a544681cc to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/58361385a4ed42098e5c4e6a544681cc 2023-07-24 12:16:42,086 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/7e6c82f848bc414ca58e79844e14cf13 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/7e6c82f848bc414ca58e79844e14cf13 2023-07-24 12:16:42,087 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/7f091c2de0c64121852ad00f8a4c34f7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/7f091c2de0c64121852ad00f8a4c34f7 2023-07-24 12:16:42,088 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/9642abacff2e478f8d68f8edfad7d2a8 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/9642abacff2e478f8d68f8edfad7d2a8 2023-07-24 12:16:42,089 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/b2b65d66e96c473dbde2a787a862feda to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/b2b65d66e96c473dbde2a787a862feda 2023-07-24 12:16:42,090 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/efadf79b50904464a9643f63d493f32c to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/efadf79b50904464a9643f63d493f32c 2023-07-24 12:16:42,091 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/f354c1248fe94d4e95b7deb9a135f78b to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/cf2/f354c1248fe94d4e95b7deb9a135f78b 2023-07-24 12:16:42,093 DEBUG [HFileArchiver-9] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/recovered.edits/85.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76/recovered.edits/85.seqid 2023-07-24 12:16:42,093 DEBUG [HFileArchiver-9] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegionEmptyDaughter/ff6186d783435ac95747fe76928d7a76 2023-07-24 12:16:42,093 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegionEmptyDaughter regions 2023-07-24 12:16:42,095 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=140, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 12:16:42,097 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegionEmptyDaughter from hbase:meta 2023-07-24 12:16:42,099 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegionEmptyDaughter' descriptor. 2023-07-24 12:16:42,100 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=140, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 12:16:42,100 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegionEmptyDaughter' from region states. 2023-07-24 12:16:42,100 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690201002100"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:42,100 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690201002100"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:42,100 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690201002100"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:42,101 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-24 12:16:42,101 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => ff6186d783435ac95747fe76928d7a76, NAME => 'testSplitTableRegionEmptyDaughter,,1690200997511.ff6186d783435ac95747fe76928d7a76.', STARTKEY => '', ENDKEY => ''}, {ENCODED => e874484af740f9942cb9359ddd22ed10, NAME => 'testSplitTableRegionEmptyDaughter,,1690201000371.e874484af740f9942cb9359ddd22ed10.', STARTKEY => '', ENDKEY => '71'}, {ENCODED => 8691b3a4f0e752b460f46e505ee0aeb4, NAME => 'testSplitTableRegionEmptyDaughter,71,1690201000371.8691b3a4f0e752b460f46e505ee0aeb4.', STARTKEY => '71', ENDKEY => ''}] 2023-07-24 12:16:42,101 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegionEmptyDaughter' as deleted. 2023-07-24 12:16:42,101 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegionEmptyDaughter","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690201002101"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:42,103 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegionEmptyDaughter state from META 2023-07-24 12:16:42,104 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=140, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter 2023-07-24 12:16:42,105 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=140, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegionEmptyDaughter in 46 msec 2023-07-24 12:16:42,106 ERROR [BP-51740533-172.31.2.20-1690200889613 heartbeating to localhost/127.0.0.1:33821] datanode.BPServiceActor(848): Exception in BPOfferService for Block pool BP-51740533-172.31.2.20-1690200889613 (Datanode Uuid 8ac734ec-00fd-4136-ae08-8d364bd1fe4b) service to localhost/127.0.0.1:33821 java.lang.IllegalThreadStateException at java.lang.ThreadGroup.addUnstarted(ThreadGroup.java:867) at java.lang.Thread.init(Thread.java:407) at java.lang.Thread.init(Thread.java:351) at java.lang.Thread.(Thread.java:501) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService$1.newThread(FsDatasetAsyncDiskService.java:102) at java.util.concurrent.ThreadPoolExecutor$Worker.(ThreadPoolExecutor.java:619) at java.util.concurrent.ThreadPoolExecutor.addWorker(ThreadPoolExecutor.java:932) at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1367) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.execute(FsDatasetAsyncDiskService.java:172) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetAsyncDiskService.deleteAsync(FsDatasetAsyncDiskService.java:221) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2216) at org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.FsDatasetImpl.invalidate(FsDatasetImpl.java:2140) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActive(BPOfferService.java:732) at org.apache.hadoop.hdfs.server.datanode.BPOfferService.processCommandFromActor(BPOfferService.java:679) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.processCommand(BPServiceActor.java:880) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.offerService(BPServiceActor.java:676) at org.apache.hadoop.hdfs.server.datanode.BPServiceActor.run(BPServiceActor.java:846) at java.lang.Thread.run(Thread.java:750) 2023-07-24 12:16:42,162 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=140 2023-07-24 12:16:42,163 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegionEmptyDaughter, procId: 140 completed 2023-07-24 12:16:42,176 INFO [Listener at localhost/37279] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegionEmptyDaughter Thread=375 (was 377), OpenFileDescriptor=839 (was 841), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=169 (was 175), ProcessCount=168 (was 168), AvailableMemoryMB=1483 (was 1488) 2023-07-24 12:16:42,188 INFO [Listener at localhost/37279] hbase.ResourceChecker(147): before: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=375, OpenFileDescriptor=839, MaxFileDescriptor=60000, SystemLoadAverage=169, ProcessCount=168, AvailableMemoryMB=1482 2023-07-24 12:16:42,188 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:16:42,190 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDataChanged, state=SyncConnected, path=/hbase/balancer 2023-07-24 12:16:42,191 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(492): Client=jenkins//172.31.2.20 set balanceSwitch=false 2023-07-24 12:16:42,192 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION; CreateTableProcedure table=testSplitTableRegion 2023-07-24 12:16:42,193 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_PRE_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_PRE_OPERATION 2023-07-24 12:16:42,194 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_WRITE_FS_LAYOUT, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_WRITE_FS_LAYOUT 2023-07-24 12:16:42,195 DEBUG [HFileArchiver-12] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:42,195 DEBUG [HFileArchiver-12] backup.HFileArchiver(153): Directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf empty. 2023-07-24 12:16:42,196 DEBUG [HFileArchiver-12] backup.HFileArchiver(599): Failed to delete directory hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:42,196 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-07-24 12:16:42,205 DEBUG [PEWorker-1] util.FSTableDescriptors(570): Wrote into hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/.tabledesc/.tableinfo.0000000001 2023-07-24 12:16:42,206 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(7675): creating {ENCODED => 62e3147f93b8eed551ff46501d4053bf, NAME => 'testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf.', STARTKEY => '', ENDKEY => ''}, tableDescriptor='testSplitTableRegion', {NAME => 'cf1', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, {NAME => 'cf2', BLOOMFILTER => 'ROW', IN_MEMORY => 'false', VERSIONS => '1', KEEP_DELETED_CELLS => 'FALSE', DATA_BLOCK_ENCODING => 'NONE', COMPRESSION => 'NONE', TTL => 'FOREVER', MIN_VERSIONS => '0', BLOCKCACHE => 'true', BLOCKSIZE => '65536', REPLICATION_SCOPE => '0'}, regionDir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp 2023-07-24 12:16:42,212 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:16:42,212 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1604): Closing 62e3147f93b8eed551ff46501d4053bf, disabling compactions & flushes 2023-07-24 12:16:42,212 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:42,212 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:42,212 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. after waiting 0 ms 2023-07-24 12:16:42,212 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:42,212 INFO [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:42,212 DEBUG [RegionOpenAndInit-testSplitTableRegion-pool-0] regionserver.HRegion(1558): Region close journal for 62e3147f93b8eed551ff46501d4053bf: 2023-07-24 12:16:42,214 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ADD_TO_META, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ADD_TO_META 2023-07-24 12:16:42,215 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":2,"row":"testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690201002214"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690201002214"}]},"ts":"1690201002214"} 2023-07-24 12:16:42,216 INFO [PEWorker-1] hbase.MetaTableAccessor(1496): Added 1 regions to meta. 2023-07-24 12:16:42,216 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_ASSIGN_REGIONS, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_ASSIGN_REGIONS 2023-07-24 12:16:42,216 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690201002216"}]},"ts":"1690201002216"} 2023-07-24 12:16:42,217 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLING in hbase:meta 2023-07-24 12:16:42,220 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(334): Hosts are {jenkins-hbase6.apache.org=0} racks are {/default-rack=0} 2023-07-24 12:16:42,221 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 0 is on host 0 2023-07-24 12:16:42,221 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 1 is on host 0 2023-07-24 12:16:42,221 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(362): server 2 is on host 0 2023-07-24 12:16:42,221 DEBUG [PEWorker-1] balancer.BaseLoadBalancer$Cluster(378): Number of tables=1, number of hosts=1, number of racks=1 2023-07-24 12:16:42,221 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=62e3147f93b8eed551ff46501d4053bf, ASSIGN}] 2023-07-24 12:16:42,222 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=62e3147f93b8eed551ff46501d4053bf, ASSIGN 2023-07-24 12:16:42,223 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=142, ppid=141, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=62e3147f93b8eed551ff46501d4053bf, ASSIGN; state=OFFLINE, location=jenkins-hbase6.apache.org,36983,1690200894572; forceNewPlan=false, retain=false 2023-07-24 12:16:42,373 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 1 regions. 1 retained the pre-restart assignment. 2023-07-24 12:16:42,374 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=62e3147f93b8eed551ff46501d4053bf, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,374 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690201002374"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690201002374"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690201002374"}]},"ts":"1690201002374"} 2023-07-24 12:16:42,375 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=143, ppid=142, state=RUNNABLE; OpenRegionProcedure 62e3147f93b8eed551ff46501d4053bf, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:16:42,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:42,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 62e3147f93b8eed551ff46501d4053bf, NAME => 'testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf.', STARTKEY => '', ENDKEY => ''} 2023-07-24 12:16:42,533 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:16:42,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:42,533 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:16:42,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:42,534 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:42,535 INFO [StoreOpener-62e3147f93b8eed551ff46501d4053bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:42,536 DEBUG [StoreOpener-62e3147f93b8eed551ff46501d4053bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1 2023-07-24 12:16:42,536 DEBUG [StoreOpener-62e3147f93b8eed551ff46501d4053bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1 2023-07-24 12:16:42,536 INFO [StoreOpener-62e3147f93b8eed551ff46501d4053bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 62e3147f93b8eed551ff46501d4053bf columnFamilyName cf1 2023-07-24 12:16:42,537 INFO [StoreOpener-62e3147f93b8eed551ff46501d4053bf-1] regionserver.HStore(310): Store=62e3147f93b8eed551ff46501d4053bf/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:42,537 INFO [StoreOpener-62e3147f93b8eed551ff46501d4053bf-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:42,538 DEBUG [StoreOpener-62e3147f93b8eed551ff46501d4053bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2 2023-07-24 12:16:42,538 DEBUG [StoreOpener-62e3147f93b8eed551ff46501d4053bf-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2 2023-07-24 12:16:42,538 INFO [StoreOpener-62e3147f93b8eed551ff46501d4053bf-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 62e3147f93b8eed551ff46501d4053bf columnFamilyName cf2 2023-07-24 12:16:42,538 INFO [StoreOpener-62e3147f93b8eed551ff46501d4053bf-1] regionserver.HStore(310): Store=62e3147f93b8eed551ff46501d4053bf/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:42,539 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:42,539 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:42,541 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:16:42,542 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:42,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/recovered.edits/1.seqid, newMaxSeqId=1, maxSeqId=-1 2023-07-24 12:16:42,544 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 62e3147f93b8eed551ff46501d4053bf; next sequenceid=2; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=74740450, jitterRate=0.11371949315071106}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:16:42,544 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 62e3147f93b8eed551ff46501d4053bf: 2023-07-24 12:16:42,545 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf., pid=143, masterSystemTime=1690201002527 2023-07-24 12:16:42,546 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:42,546 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:42,546 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=142 updating hbase:meta row=62e3147f93b8eed551ff46501d4053bf, regionState=OPEN, openSeqNum=2, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,547 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690201002546"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690201002546"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690201002546"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690201002546"}]},"ts":"1690201002546"} 2023-07-24 12:16:42,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=143, resume processing ppid=142 2023-07-24 12:16:42,549 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=143, ppid=142, state=SUCCESS; OpenRegionProcedure 62e3147f93b8eed551ff46501d4053bf, server=jenkins-hbase6.apache.org,36983,1690200894572 in 173 msec 2023-07-24 12:16:42,550 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=142, resume processing ppid=141 2023-07-24 12:16:42,550 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=142, ppid=141, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=62e3147f93b8eed551ff46501d4053bf, ASSIGN in 328 msec 2023-07-24 12:16:42,551 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_UPDATE_DESC_CACHE, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_UPDATE_DESC_CACHE 2023-07-24 12:16:42,551 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690201002551"}]},"ts":"1690201002551"} 2023-07-24 12:16:42,552 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=ENABLED in hbase:meta 2023-07-24 12:16:42,554 INFO [PEWorker-1] procedure.CreateTableProcedure(80): pid=141, state=RUNNABLE:CREATE_TABLE_POST_OPERATION, locked=true; CreateTableProcedure table=testSplitTableRegion execute state=CREATE_TABLE_POST_OPERATION 2023-07-24 12:16:42,555 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=141, state=SUCCESS; CreateTableProcedure table=testSplitTableRegion in 363 msec 2023-07-24 12:16:42,699 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:42,701 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-24 12:16:42,701 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-24 12:16:42,701 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:42,701 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-24 12:16:42,701 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-24 12:16:42,701 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,701 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,702 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:42,702 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:42,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:42,702 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,702 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:42,702 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:42,702 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:42,703 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:42,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:42,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:42,703 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,703 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:42,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,703 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,703 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:42,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,704 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:42,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:42,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:42,704 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,704 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:42,704 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:42,704 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:42,704 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:42,704 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:42,704 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:42,704 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:42,704 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:42,704 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:42,704 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:42,704 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:42,704 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:42,704 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:42,704 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:42,704 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:42,705 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:42,705 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:42,705 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:42,705 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:42,705 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:42,705 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:42,705 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. started... 2023-07-24 12:16:42,706 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 62e3147f93b8eed551ff46501d4053bf 2/2 column families, dataSize=120 B heapSize=928 B 2023-07-24 12:16:42,707 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,708 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,708 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,708 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:42,708 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,708 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:42,715 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf1/91feaa72255640fca41373b22989b49d 2023-07-24 12:16:42,725 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=60 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf2/3552735b9780482baaaf6e26ae679da8 2023-07-24 12:16:42,729 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf1/91feaa72255640fca41373b22989b49d as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/91feaa72255640fca41373b22989b49d 2023-07-24 12:16:42,732 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/91feaa72255640fca41373b22989b49d, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:16:42,732 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf2/3552735b9780482baaaf6e26ae679da8 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/3552735b9780482baaaf6e26ae679da8 2023-07-24 12:16:42,735 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/3552735b9780482baaaf6e26ae679da8, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:16:42,736 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~120 B/120, heapSize ~896 B/896, currentSize=0 B/0 for 62e3147f93b8eed551ff46501d4053bf in 30ms, sequenceid=6, compaction requested=false 2023-07-24 12:16:42,736 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.MetricsTableSourceImpl(130): Creating new MetricsTableSourceImpl for table 'testSplitTableRegion' 2023-07-24 12:16:42,737 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 62e3147f93b8eed551ff46501d4053bf: 2023-07-24 12:16:42,737 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:42,737 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:42,737 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:42,737 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,737 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:42,737 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:42,738 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,738 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,738 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:42,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:42,739 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,739 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:42,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:42,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:42,739 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:42,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,740 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:42,740 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-24 12:16:42,741 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@18994b1b[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:42,741 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-24 12:16:42,741 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,742 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,742 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,742 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,742 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:42,742 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,742 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:42,742 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:42,742 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,742 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:42,742 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,742 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:42,742 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:42,742 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:42,742 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:42,743 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:42,743 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:42,747 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,747 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:42,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:42,747 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:42,747 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:42,747 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:42,750 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:42,750 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:42,750 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:42,750 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:42,751 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:42,751 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:42,750 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:42,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:42,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:42,751 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:42,752 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:42,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,753 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:16:42,754 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-24 12:16:42,754 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-24 12:16:42,754 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:42,754 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:42,754 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-24 12:16:42,754 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:42,755 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,755 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,755 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,755 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:42,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,755 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,755 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,755 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:42,755 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:42,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,756 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:42,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:42,756 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:42,756 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:42,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:42,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:42,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,756 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,757 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:42,756 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:42,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:42,757 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:42,757 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:42,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:42,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,758 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,758 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,759 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:42,759 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:42,759 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,760 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,763 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,763 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:42,763 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,763 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:42,763 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,763 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:42,763 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:42,763 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:42,763 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:42,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:42,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:42,763 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:42,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-24 12:16:42,763 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:42,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:42,763 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,763 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:42,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:42,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:42,763 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:42,763 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:42,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:42,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:42,763 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,764 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:42,763 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:42,764 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,764 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,764 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,764 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,764 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,764 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:42,764 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:42,864 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:42,865 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:42,874 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:42,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:42,875 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-24 12:16:42,875 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-24 12:16:42,875 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:42,875 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-24 12:16:42,875 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-24 12:16:42,876 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,876 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,877 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:42,877 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:42,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:42,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:42,877 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,877 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:42,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:42,877 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:42,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:42,877 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:42,877 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,878 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:42,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,878 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,878 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:42,878 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:42,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,878 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,878 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:42,878 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:42,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:42,879 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,879 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:42,879 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:42,879 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:42,879 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:42,879 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:42,879 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:42,879 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:42,879 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:42,879 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:42,879 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:42,879 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:42,879 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:42,879 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:42,879 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:42,879 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:42,879 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:42,879 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:42,879 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:42,879 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:42,880 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:42,880 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:42,880 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:42,880 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:42,880 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. started... 2023-07-24 12:16:42,880 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 62e3147f93b8eed551ff46501d4053bf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:42,881 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,882 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,882 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,882 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:42,882 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,882 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:42,892 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf1/6c423cf05c0e4689b54c93f830269180 2023-07-24 12:16:42,903 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=19 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf2/e5aba3df13884f5399214e0bfb45d1c2 2023-07-24 12:16:42,907 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf1/6c423cf05c0e4689b54c93f830269180 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/6c423cf05c0e4689b54c93f830269180 2023-07-24 12:16:42,911 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/6c423cf05c0e4689b54c93f830269180, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:16:42,911 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf2/e5aba3df13884f5399214e0bfb45d1c2 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/e5aba3df13884f5399214e0bfb45d1c2 2023-07-24 12:16:42,914 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/e5aba3df13884f5399214e0bfb45d1c2, entries=10, sequenceid=19, filesize=5.0 K 2023-07-24 12:16:42,915 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 62e3147f93b8eed551ff46501d4053bf in 35ms, sequenceid=19, compaction requested=false 2023-07-24 12:16:42,915 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 62e3147f93b8eed551ff46501d4053bf: 2023-07-24 12:16:42,915 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:42,915 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:42,915 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:42,915 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,915 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:42,915 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:42,918 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,918 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:42,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:42,918 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,918 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:42,918 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:42,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:42,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:42,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,919 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,920 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:42,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-24 12:16:42,920 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@99f890c[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:42,920 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-24 12:16:42,920 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,921 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,921 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,921 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,921 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,921 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:42,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,922 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,922 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:42,922 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:42,921 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,922 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:42,922 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:42,922 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:42,922 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:42,922 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,922 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:42,922 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:42,922 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:42,923 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:42,923 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:42,923 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:42,924 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:42,924 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:42,924 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,924 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:42,924 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:42,924 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:42,924 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:42,924 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,925 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:42,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:42,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,926 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,927 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:16:42,927 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-24 12:16:42,927 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-24 12:16:42,927 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:42,927 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:42,927 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-24 12:16:42,927 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:42,929 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,929 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,929 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,929 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:42,929 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:42,929 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,929 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:42,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:42,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:42,929 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,929 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:42,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:42,930 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:42,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:42,930 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:42,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:42,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:42,930 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,930 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:42,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:42,931 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,931 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:42,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:42,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,932 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,933 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:42,933 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:42,933 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,933 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,934 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:42,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:42,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:42,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:42,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:42,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-24 12:16:42,936 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:42,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:42,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:42,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:42,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:42,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:42,936 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:42,936 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:42,936 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:42,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:42,936 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:42,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:42,936 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:42,936 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:42,937 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:42,937 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,937 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:42,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:42,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:42,937 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:42,937 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:42,937 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:42,937 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:42,937 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,036 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:43,037 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:43,047 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:43,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:43,049 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-24 12:16:43,049 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-24 12:16:43,049 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,049 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-24 12:16:43,049 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-24 12:16:43,049 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,049 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,051 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,051 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,051 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,051 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,051 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,051 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,051 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:43,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,051 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,051 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,052 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:43,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:43,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:43,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:43,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:43,052 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,052 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:43,052 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:43,052 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:43,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:43,052 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:43,052 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:43,052 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:43,052 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:43,053 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:43,053 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,052 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:43,053 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:43,053 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:43,053 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,053 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:43,053 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:43,053 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:43,053 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:43,053 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:43,053 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:43,053 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:43,053 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:43,053 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. started... 2023-07-24 12:16:43,053 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 62e3147f93b8eed551ff46501d4053bf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:43,063 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,063 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,063 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,063 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,063 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:43,063 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:43,067 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf1/65b5fe2ba4194b79b8986167053580bc 2023-07-24 12:16:43,076 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=32 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf2/c7ab253de060407daadfee44e0e34624 2023-07-24 12:16:43,079 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf1/65b5fe2ba4194b79b8986167053580bc as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/65b5fe2ba4194b79b8986167053580bc 2023-07-24 12:16:43,082 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/65b5fe2ba4194b79b8986167053580bc, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:16:43,083 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf2/c7ab253de060407daadfee44e0e34624 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/c7ab253de060407daadfee44e0e34624 2023-07-24 12:16:43,086 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/c7ab253de060407daadfee44e0e34624, entries=10, sequenceid=32, filesize=5.0 K 2023-07-24 12:16:43,086 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 62e3147f93b8eed551ff46501d4053bf in 33ms, sequenceid=32, compaction requested=true 2023-07-24 12:16:43,086 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 62e3147f93b8eed551ff46501d4053bf: 2023-07-24 12:16:43,086 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:43,086 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:43,086 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:43,086 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,087 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:43,087 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:43,088 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,088 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:43,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:43,088 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,088 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:43,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:43,088 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:43,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,089 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,090 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:43,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-24 12:16:43,090 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@91eac1b[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:43,090 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-24 12:16:43,090 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,091 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,091 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,091 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,091 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,091 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,091 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:43,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,091 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,092 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:43,092 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:43,091 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:43,092 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:43,092 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:43,092 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:43,092 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:43,092 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:43,092 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:43,093 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,093 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:43,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:43,093 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:43,093 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,093 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:43,095 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:43,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:43,095 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:43,095 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,095 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,095 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:43,095 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:43,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:43,095 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:43,096 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,097 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,098 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:16:43,098 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-24 12:16:43,098 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-24 12:16:43,098 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:43,098 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:43,098 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-24 12:16:43,098 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:43,099 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,099 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,099 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,099 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:43,099 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:43,099 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,099 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,099 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,100 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,100 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:43,100 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,100 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,100 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,101 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:43,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,101 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,102 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,102 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:43,103 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,103 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,103 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,105 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,105 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,105 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,105 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,105 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,106 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,106 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,105 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,105 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,105 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,106 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-24 12:16:43,106 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:43,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,106 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,106 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,106 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,106 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:43,106 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:43,106 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,106 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,107 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,106 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,107 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,107 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,107 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,107 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,207 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:43,207 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:43,217 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:43,218 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:43,219 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-24 12:16:43,219 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-24 12:16:43,219 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,219 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-24 12:16:43,219 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-24 12:16:43,219 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,219 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,220 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,220 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,220 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,220 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,221 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,221 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:43,220 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,221 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,221 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:43,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,222 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:43,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:43,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:43,222 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:43,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,222 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:43,222 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,222 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:43,222 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:43,222 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,222 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:43,222 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:43,222 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:43,222 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:43,222 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:43,222 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:43,222 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:43,222 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:43,222 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:43,222 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,223 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:43,223 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:43,223 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,223 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:43,223 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:43,223 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:43,223 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:43,223 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:43,223 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:43,223 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. started... 2023-07-24 12:16:43,223 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 62e3147f93b8eed551ff46501d4053bf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:43,224 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,225 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,225 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:43,225 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,226 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,226 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:43,231 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf1/d2111213b7c54ad581aa89ab4418b302 2023-07-24 12:16:43,241 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=45 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf2/2714ecf62de54291ac8e9265bffe6eb0 2023-07-24 12:16:43,245 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf1/d2111213b7c54ad581aa89ab4418b302 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/d2111213b7c54ad581aa89ab4418b302 2023-07-24 12:16:43,248 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/d2111213b7c54ad581aa89ab4418b302, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:16:43,249 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf2/2714ecf62de54291ac8e9265bffe6eb0 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/2714ecf62de54291ac8e9265bffe6eb0 2023-07-24 12:16:43,252 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/2714ecf62de54291ac8e9265bffe6eb0, entries=10, sequenceid=45, filesize=5.0 K 2023-07-24 12:16:43,252 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 62e3147f93b8eed551ff46501d4053bf in 29ms, sequenceid=45, compaction requested=true 2023-07-24 12:16:43,253 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 62e3147f93b8eed551ff46501d4053bf: 2023-07-24 12:16:43,253 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:43,253 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:43,253 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:43,253 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,253 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:43,253 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:43,254 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,254 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:43,254 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:43,255 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:43,255 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:43,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:43,255 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,256 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:43,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-24 12:16:43,257 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@48369b05[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:43,257 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-24 12:16:43,257 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,258 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,258 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,258 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,258 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,258 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,258 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,258 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:43,258 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:43,258 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:43,258 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:43,258 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:43,258 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:43,258 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:43,258 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:43,259 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:43,259 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:43,260 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:43,261 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,261 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:43,261 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,261 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:43,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:43,261 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:43,261 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:43,261 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,261 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,262 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:43,262 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:43,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:43,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:43,262 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,263 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:43,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,264 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,265 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:16:43,265 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-24 12:16:43,265 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-24 12:16:43,265 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:43,265 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:43,265 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-24 12:16:43,265 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:43,267 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,267 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,267 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,267 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,267 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:43,267 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:43,267 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,267 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,268 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:43,268 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,268 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,269 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,269 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,269 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:43,270 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,270 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,271 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:43,272 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,272 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,272 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,273 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,273 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,279 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,279 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,279 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,279 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,279 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,279 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,279 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,279 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,280 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,280 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,280 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,279 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-24 12:16:43,280 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:43,279 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,279 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,280 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,280 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,280 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:43,280 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:43,280 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,281 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,280 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,281 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,281 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,281 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,281 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,281 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,381 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:43,381 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:43,392 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:43,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:43,393 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-24 12:16:43,393 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-24 12:16:43,393 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,393 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-24 12:16:43,393 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-24 12:16:43,393 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,394 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,395 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,395 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,395 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,395 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,395 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,395 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,395 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,395 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,396 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:43,396 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:43,396 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:43,397 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,397 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:43,397 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:43,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:43,397 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:43,397 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:43,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:43,397 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,397 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:43,397 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:43,397 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:43,397 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:43,397 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,397 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:43,397 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:43,397 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:43,397 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,397 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:43,397 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:43,397 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:43,397 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:43,397 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:43,398 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:43,398 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:43,398 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. started... 2023-07-24 12:16:43,398 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 62e3147f93b8eed551ff46501d4053bf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:43,399 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,399 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,399 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,399 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:43,399 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,399 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:43,407 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf1/24600f1518f34667bfa83861f16713e9 2023-07-24 12:16:43,417 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=58 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf2/24660271f70d4dd2a323b9099dacc4f7 2023-07-24 12:16:43,420 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf1/24600f1518f34667bfa83861f16713e9 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/24600f1518f34667bfa83861f16713e9 2023-07-24 12:16:43,423 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/24600f1518f34667bfa83861f16713e9, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:16:43,423 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf2/24660271f70d4dd2a323b9099dacc4f7 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/24660271f70d4dd2a323b9099dacc4f7 2023-07-24 12:16:43,426 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/24660271f70d4dd2a323b9099dacc4f7, entries=10, sequenceid=58, filesize=5.0 K 2023-07-24 12:16:43,427 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 62e3147f93b8eed551ff46501d4053bf in 29ms, sequenceid=58, compaction requested=true 2023-07-24 12:16:43,427 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 62e3147f93b8eed551ff46501d4053bf: 2023-07-24 12:16:43,427 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:43,427 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:43,427 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:43,427 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,427 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:43,427 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:43,429 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,429 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:43,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:43,429 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,429 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:43,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:43,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:43,429 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,430 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:43,431 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-24 12:16:43,431 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@1aa5d08b[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:43,431 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-24 12:16:43,431 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,437 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,437 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,437 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,437 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,437 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:43,437 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,437 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:43,437 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:43,437 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:43,437 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:43,438 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:43,438 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:43,437 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,438 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,438 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:43,438 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:43,438 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:43,439 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:43,439 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,439 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:43,440 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:43,440 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,440 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,440 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:43,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:43,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:43,440 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:43,440 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,440 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:43,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:43,440 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:43,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,441 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:43,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,442 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,443 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:16:43,443 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-24 12:16:43,443 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-24 12:16:43,443 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:43,443 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:43,443 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-24 12:16:43,443 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:43,444 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,444 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,444 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,444 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,444 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,444 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,444 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,444 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,445 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,445 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:43,445 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,445 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,445 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:43,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,446 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,446 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:43,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,447 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,448 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:43,448 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,448 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,448 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,449 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,452 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,452 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,452 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,452 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,452 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,452 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,453 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,453 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,453 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,452 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-24 12:16:43,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,452 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,453 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:43,453 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,453 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,453 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,453 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:43,453 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,453 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,453 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:43,454 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,454 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,454 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,454 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,454 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,454 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,454 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,454 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,554 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:43,554 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:43,565 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(936): Client=jenkins//172.31.2.20 procedure request for: flush-table-proc 2023-07-24 12:16:43,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(143): Procedure testSplitTableRegion was in running list but was completed. Accepting new attempt. 2023-07-24 12:16:43,567 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure.ProcedureCoordinator(165): Submitting procedure testSplitTableRegion 2023-07-24 12:16:43,567 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(191): Starting procedure 'testSplitTableRegion' 2023-07-24 12:16:43,567 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,567 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(199): Procedure 'testSplitTableRegion' starting 'acquire' 2023-07-24 12:16:43,567 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(241): Starting procedure 'testSplitTableRegion', kicking off acquire phase on members. 2023-07-24 12:16:43,567 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,567 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(92): Creating acquire znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,569 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,569 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,569 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,569 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(100): Watching for acquire node:/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,569 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,569 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,569 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(203): Waiting for all members to 'acquire' 2023-07-24 12:16:43,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,569 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(186): Found procedure znode: /hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,570 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(212): start proc data length is 4 2023-07-24 12:16:43,570 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(214): Found data for znode:/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:43,570 DEBUG [zk-event-processor-pool-0] flush.RegionServerFlushTableProcedureManager(153): Launching subprocedure to flush regions for testSplitTableRegion 2023-07-24 12:16:43,570 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:43,571 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:43,571 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:43,571 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:43,571 WARN [zk-event-processor-pool-0] procedure.ProcedureMember(133): A completed old subproc testSplitTableRegion is still present, removing 2023-07-24 12:16:43,571 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,571 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:43,571 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:43,571 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:43,571 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,571 DEBUG [zk-event-processor-pool-0] procedure.ProcedureMember(140): Submitting new Subprocedure:testSplitTableRegion 2023-07-24 12:16:43,571 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:43,571 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:43,571 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,34575,1690200894532' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:43,571 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:43,571 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(151): Starting subprocedure 'testSplitTableRegion' with timeout 60000ms 2023-07-24 12:16:43,571 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(107): Scheduling process timer to run in: 60000 ms 2023-07-24 12:16:43,571 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,41789,1690200894488' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:43,571 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(159): Subprocedure 'testSplitTableRegion' starting 'acquire' stage 2023-07-24 12:16:43,571 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.FlushTableSubprocedure(113): Flush region tasks submitted for 1 regions 2023-07-24 12:16:43,572 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(242): Waiting for local region flush to finish. 2023-07-24 12:16:43,572 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(69): Starting region operation on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:43,572 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(72): Flush region testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. started... 2023-07-24 12:16:43,572 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2745): Flushing 62e3147f93b8eed551ff46501d4053bf 2/2 column families, dataSize=600 B heapSize=2.53 KB 2023-07-24 12:16:43,573 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,574 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,574 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:43,574 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,574 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,574 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:43,580 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf1/45f2aabb72ed46698703826e2988bdb4 2023-07-24 12:16:43,592 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=300 B at sequenceid=71 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf2/330cd48cce594c7b98ec0f5724297c5e 2023-07-24 12:16:43,595 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf1/45f2aabb72ed46698703826e2988bdb4 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/45f2aabb72ed46698703826e2988bdb4 2023-07-24 12:16:43,598 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/45f2aabb72ed46698703826e2988bdb4, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:16:43,599 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf2/330cd48cce594c7b98ec0f5724297c5e as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/330cd48cce594c7b98ec0f5724297c5e 2023-07-24 12:16:43,602 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/330cd48cce594c7b98ec0f5724297c5e, entries=10, sequenceid=71, filesize=5.0 K 2023-07-24 12:16:43,602 INFO [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2948): Finished flush of dataSize ~600 B/600, heapSize ~2.50 KB/2560, currentSize=0 B/0 for 62e3147f93b8eed551ff46501d4053bf in 30ms, sequenceid=71, compaction requested=true 2023-07-24 12:16:43,602 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] regionserver.HRegion(2446): Flush status journal for 62e3147f93b8eed551ff46501d4053bf: 2023-07-24 12:16:43,602 DEBUG [rs(jenkins-hbase6.apache.org,36983,1690200894572)-flush-proc-pool-0] flush.FlushTableSubprocedure$RegionFlushTask(80): Closing region operation on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:43,602 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(253): Completed 1/1 local region flush tasks. 2023-07-24 12:16:43,602 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(255): Completed 1 local region flush tasks. 2023-07-24 12:16:43,603 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] flush.RegionServerFlushTableProcedureManager$FlushTableSubprocedurePool(287): cancelling 0 flush region tasks jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,603 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(161): Subprocedure 'testSplitTableRegion' locally acquired 2023-07-24 12:16:43,603 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(242): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure (testSplitTableRegion) in zk 2023-07-24 12:16:43,605 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,605 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(250): Watch for global barrier reached:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:43,605 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:43,606 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,606 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(166): Subprocedure 'testSplitTableRegion' coordinator notified of 'acquire', waiting on 'reached' or 'abort' from coordinator 2023-07-24 12:16:43,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:43,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:43,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,606 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,607 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:43,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(291): member: 'jenkins-hbase6.apache.org,36983,1690200894572' joining acquired barrier for procedure 'testSplitTableRegion' on coordinator 2023-07-24 12:16:43,608 DEBUG [zk-event-processor-pool-0] procedure.Procedure(300): Waiting on: java.util.concurrent.CountDownLatch@6f07fd3a[Count = 0] remaining members to acquire global barrier 2023-07-24 12:16:43,608 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(207): Procedure 'testSplitTableRegion' starting 'in-barrier' execution. 2023-07-24 12:16:43,608 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(116): Creating reached barrier zk node:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,609 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,609 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,609 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,609 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,609 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,609 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(211): Waiting for all members to 'release' 2023-07-24 12:16:43,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,609 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(128): Received reached global barrier:/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,609 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:43,609 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:43,609 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:43,609 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:43,609 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,41789,1690200894488' in zk 2023-07-24 12:16:43,609 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(180): Subprocedure 'testSplitTableRegion' received 'reached' from coordinator. 2023-07-24 12:16:43,609 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(182): Subprocedure 'testSplitTableRegion' locally completed 2023-07-24 12:16:43,609 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,36983,1690200894572' in zk 2023-07-24 12:16:43,609 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.ZKProcedureMemberRpcs(267): Marking procedure 'testSplitTableRegion' completed for member 'jenkins-hbase6.apache.org,34575,1690200894532' in zk 2023-07-24 12:16:43,613 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,613 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:43,613 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,613 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,613 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:43,614 DEBUG [member: 'jenkins-hbase6.apache.org,36983,1690200894572' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:43,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:43,614 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:43,614 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,614 DEBUG [member: 'jenkins-hbase6.apache.org,41789,1690200894488' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:43,615 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(187): Subprocedure 'testSplitTableRegion' has notified controller of completion 2023-07-24 12:16:43,615 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,615 DEBUG [member: 'jenkins-hbase6.apache.org,34575,1690200894532' subprocedure-pool-0] procedure.Subprocedure(212): Subprocedure 'testSplitTableRegion' completed. 2023-07-24 12:16:43,615 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,616 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:43,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,617 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,618 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(218): Finished data from procedure 'testSplitTableRegion' member 'jenkins-hbase6.apache.org,36983,1690200894572': 2023-07-24 12:16:43,618 DEBUG [zk-event-processor-pool-0] procedure.Procedure(321): Member: 'jenkins-hbase6.apache.org,36983,1690200894572' released barrier for procedure'testSplitTableRegion', counting down latch. Waiting for 0 more 2023-07-24 12:16:43,618 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(216): Procedure 'testSplitTableRegion' execution completed 2023-07-24 12:16:43,618 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(225): Running finish phase. 2023-07-24 12:16:43,618 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.Procedure(275): Finished coordinator procedure - removing self from list of running procedures 2023-07-24 12:16:43,618 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureCoordinator(162): Attempting to clean out zk node for op:testSplitTableRegion 2023-07-24 12:16:43,618 INFO [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] procedure.ZKProcedureUtil(265): Clearing all znodes for procedure testSplitTableRegionincluding nodes /hbase/flush-table-proc/acquired /hbase/flush-table-proc/reached /hbase/flush-table-proc/abort 2023-07-24 12:16:43,620 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,620 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,620 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,620 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,620 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,620 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeCreated, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureCoordinator$1(194): Node created: /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,620 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(77): Received created event:/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,620 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,621 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,620 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(225): Current zk system: 2023-07-24 12:16:43,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(227): |-/hbase/flush-table-proc 2023-07-24 12:16:43,621 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,621 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-abort 2023-07-24 12:16:43,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,621 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(316): Aborting procedure member for znode /hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,622 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-acquired 2023-07-24 12:16:43,622 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,623 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,623 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,624 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-reached 2023-07-24 12:16:43,624 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] zookeeper.ZKUtil(162): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on existing znode=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |----testSplitTableRegion 2023-07-24 12:16:43,624 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,625 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureUtil(244): |-------jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,627 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,627 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,627 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,627 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,627 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,627 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,628 DEBUG [(jenkins-hbase6.apache.org,44983,1690200893288)-proc-coordinator-pool-0] errorhandling.TimeoutExceptionInjector(87): Marking timer as complete - no error notifications will be received for this timer. 2023-07-24 12:16:43,628 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,628 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(180): Done waiting - exec procedure flush-table-proc for 'testSplitTableRegion' 2023-07-24 12:16:43,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,627 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/acquired 2023-07-24 12:16:43,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,628 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] flush.MasterFlushTableProcedureManager(182): Master flush table procedure is successful! 2023-07-24 12:16:43,628 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,628 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(104): Received procedure start children changed event: /hbase/flush-table-proc/acquired 2023-07-24 12:16:43,628 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(154): Looking for new procedures under znode:'/hbase/flush-table-proc/acquired' 2023-07-24 12:16:43,628 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/abort 2023-07-24 12:16:43,628 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/acquired/testSplitTableRegion 2023-07-24 12:16:43,629 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2690): Waiting a max of 300000 ms for procedure 'flush-table-proc : testSplitTableRegion'' to complete. (max 20000 ms per retry) 2023-07-24 12:16:43,629 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2698): (#1) Sleeping: 100ms while waiting for procedure completion. 2023-07-24 12:16:43,629 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,629 INFO [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs$1(107): Received procedure abort children changed event: /hbase/flush-table-proc/abort 2023-07-24 12:16:43,629 DEBUG [zk-event-processor-pool-0] procedure.ZKProcedureMemberRpcs(134): Checking for aborted procedures on node: '/hbase/flush-table-proc/abort' 2023-07-24 12:16:43,629 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,629 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:43,629 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:43,629 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/reached/testSplitTableRegion 2023-07-24 12:16:43,629 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/flush-table-proc/abort/testSplitTableRegion 2023-07-24 12:16:43,729 DEBUG [Listener at localhost/37279] client.HBaseAdmin(2704): Getting current status of procedure from master... 2023-07-24 12:16:43,730 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1189): Checking to see if procedure from request:flush-table-proc is done 2023-07-24 12:16:43,736 DEBUG [Listener at localhost/37279] assignment.SplitTableRegionProcedure(219): Splittable=true state=OPEN, location=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,737 DEBUG [Listener at localhost/37279] procedure2.ProcedureExecutor(1029): Stored pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=62e3147f93b8eed551ff46501d4053bf, daughterA=c6c1c9ab03c5305a97a77e2634a5225c, daughterB=8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:43,738 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=62e3147f93b8eed551ff46501d4053bf, daughterA=c6c1c9ab03c5305a97a77e2634a5225c, daughterB=8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:43,738 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=62e3147f93b8eed551ff46501d4053bf, daughterA=c6c1c9ab03c5305a97a77e2634a5225c, daughterB=8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:43,738 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=144, state=RUNNABLE:SPLIT_TABLE_REGION_PREPARE; SplitTableRegionProcedure table=testSplitTableRegion, parent=62e3147f93b8eed551ff46501d4053bf, daughterA=c6c1c9ab03c5305a97a77e2634a5225c, daughterB=8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:43,739 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=62e3147f93b8eed551ff46501d4053bf, UNASSIGN}] 2023-07-24 12:16:43,740 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=145, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=62e3147f93b8eed551ff46501d4053bf, UNASSIGN 2023-07-24 12:16:43,741 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=62e3147f93b8eed551ff46501d4053bf, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:43,741 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690201003741"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690201003741"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690201003741"}]},"ts":"1690201003741"} 2023-07-24 12:16:43,742 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=146, ppid=145, state=RUNNABLE; CloseRegionProcedure 62e3147f93b8eed551ff46501d4053bf, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:16:43,754 WARN [HBase-Metrics2-1] impl.MetricsConfig(128): Cannot locate configuration: tried hadoop-metrics2-hbase.properties,hadoop-metrics2.properties 2023-07-24 12:16:43,893 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:43,893 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 62e3147f93b8eed551ff46501d4053bf, disabling compactions & flushes 2023-07-24 12:16:43,893 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:43,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:43,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. after waiting 0 ms 2023-07-24 12:16:43,894 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:43,894 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2745): Flushing 62e3147f93b8eed551ff46501d4053bf 2/2 column families, dataSize=480 B heapSize=2.13 KB 2023-07-24 12:16:44,302 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf1/0783f9a185a04aa795beb02ada67f44e 2023-07-24 12:16:44,313 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=240 B at sequenceid=82 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf2/a51cdb73005e402a8c2b8286e7a213c7 2023-07-24 12:16:44,317 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf1/0783f9a185a04aa795beb02ada67f44e as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/0783f9a185a04aa795beb02ada67f44e 2023-07-24 12:16:44,319 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/0783f9a185a04aa795beb02ada67f44e, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 12:16:44,320 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/.tmp/cf2/a51cdb73005e402a8c2b8286e7a213c7 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/a51cdb73005e402a8c2b8286e7a213c7 2023-07-24 12:16:44,323 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/a51cdb73005e402a8c2b8286e7a213c7, entries=8, sequenceid=82, filesize=5.0 K 2023-07-24 12:16:44,324 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~480 B/480, heapSize ~2.09 KB/2144, currentSize=0 B/0 for 62e3147f93b8eed551ff46501d4053bf in 430ms, sequenceid=82, compaction requested=true 2023-07-24 12:16:44,327 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=1 2023-07-24 12:16:44,328 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:16:44,329 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. 2023-07-24 12:16:44,329 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 62e3147f93b8eed551ff46501d4053bf: 2023-07-24 12:16:44,330 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,330 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=145 updating hbase:meta row=62e3147f93b8eed551ff46501d4053bf, regionState=CLOSED 2023-07-24 12:16:44,330 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690201004330"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690201004330"}]},"ts":"1690201004330"} 2023-07-24 12:16:44,332 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=146, resume processing ppid=145 2023-07-24 12:16:44,333 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=146, ppid=145, state=SUCCESS; CloseRegionProcedure 62e3147f93b8eed551ff46501d4053bf, server=jenkins-hbase6.apache.org,36983,1690200894572 in 589 msec 2023-07-24 12:16:44,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=145, resume processing ppid=144 2023-07-24 12:16:44,334 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=145, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=62e3147f93b8eed551ff46501d4053bf, UNASSIGN in 594 msec 2023-07-24 12:16:44,343 INFO [PEWorker-1] assignment.SplitTableRegionProcedure(694): pid=144 splitting 14 storefiles, region=62e3147f93b8eed551ff46501d4053bf, threads=14 2023-07-24 12:16:44,343 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/24660271f70d4dd2a323b9099dacc4f7 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,343 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/2714ecf62de54291ac8e9265bffe6eb0 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,343 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/330cd48cce594c7b98ec0f5724297c5e for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,343 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/3552735b9780482baaaf6e26ae679da8 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,343 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/a51cdb73005e402a8c2b8286e7a213c7 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,344 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/c7ab253de060407daadfee44e0e34624 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,344 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/e5aba3df13884f5399214e0bfb45d1c2 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,344 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/0783f9a185a04aa795beb02ada67f44e for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,344 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/24600f1518f34667bfa83861f16713e9 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,344 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/45f2aabb72ed46698703826e2988bdb4 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,344 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/65b5fe2ba4194b79b8986167053580bc for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,344 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/6c423cf05c0e4689b54c93f830269180 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,344 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/91feaa72255640fca41373b22989b49d for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,344 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(776): pid=144 splitting started for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/d2111213b7c54ad581aa89ab4418b302 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,657 DEBUG [StoreFileSplitter-pool-0] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/24660271f70d4dd2a323b9099dacc4f7 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,680 DEBUG [StoreFileSplitter-pool-12] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/91feaa72255640fca41373b22989b49d for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,688 DEBUG [StoreFileSplitter-pool-3] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/3552735b9780482baaaf6e26ae679da8 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,700 DEBUG [StoreFileSplitter-pool-11] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/6c423cf05c0e4689b54c93f830269180 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,716 DEBUG [StoreFileSplitter-pool-5] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/c7ab253de060407daadfee44e0e34624 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,716 DEBUG [StoreFileSplitter-pool-1] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/2714ecf62de54291ac8e9265bffe6eb0 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,723 DEBUG [StoreFileSplitter-pool-2] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/330cd48cce594c7b98ec0f5724297c5e for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,724 DEBUG [StoreFileSplitter-pool-4] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/a51cdb73005e402a8c2b8286e7a213c7 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,729 DEBUG [StoreFileSplitter-pool-8] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/24600f1518f34667bfa83861f16713e9 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,729 DEBUG [StoreFileSplitter-pool-9] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/45f2aabb72ed46698703826e2988bdb4 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,736 DEBUG [StoreFileSplitter-pool-10] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/65b5fe2ba4194b79b8986167053580bc for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,736 DEBUG [StoreFileSplitter-pool-13] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/d2111213b7c54ad581aa89ab4418b302 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,736 DEBUG [StoreFileSplitter-pool-6] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/e5aba3df13884f5399214e0bfb45d1c2 for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,738 DEBUG [StoreFileSplitter-pool-7] assignment.SplitTableRegionProcedure(787): pid=144 splitting complete for store file: hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/0783f9a185a04aa795beb02ada67f44e for region: 62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:44,738 DEBUG [PEWorker-1] assignment.SplitTableRegionProcedure(755): pid=144 split storefiles for region 62e3147f93b8eed551ff46501d4053bf Daughter A: 14 storefiles, Daughter B: 14 storefiles. 2023-07-24 12:16:44,779 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 12:16:44,781 DEBUG [PEWorker-1] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/recovered.edits/85.seqid, newMaxSeqId=85, maxSeqId=-1 2023-07-24 12:16:44,782 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf.","families":{"info":[{"qualifier":"regioninfo","vlen":54,"tag":[],"timestamp":"1690201004782"},{"qualifier":"splitA","vlen":56,"tag":[],"timestamp":"1690201004782"},{"qualifier":"splitB","vlen":56,"tag":[],"timestamp":"1690201004782"}]},"ts":"1690201004782"} 2023-07-24 12:16:44,782 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690201004782"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690201004782"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690201004782"}]},"ts":"1690201004782"} 2023-07-24 12:16:44,782 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690201004782"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690201004782"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690201004782"}]},"ts":"1690201004782"} 2023-07-24 12:16:44,785 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=c6c1c9ab03c5305a97a77e2634a5225c, ASSIGN}, {pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=8adba019c41c42da557e82cb7e9bb78f, ASSIGN}] 2023-07-24 12:16:44,786 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=8adba019c41c42da557e82cb7e9bb78f, ASSIGN 2023-07-24 12:16:44,786 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=148, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=8adba019c41c42da557e82cb7e9bb78f, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase6.apache.org,36983,1690200894572; forceNewPlan=false, retain=false 2023-07-24 12:16:44,787 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE; TransitRegionStateProcedure table=testSplitTableRegion, region=c6c1c9ab03c5305a97a77e2634a5225c, ASSIGN 2023-07-24 12:16:44,787 INFO [PEWorker-1] assignment.TransitRegionStateProcedure(193): Starting pid=147, ppid=144, state=RUNNABLE:REGION_STATE_TRANSITION_GET_ASSIGN_CANDIDATE, locked=true; TransitRegionStateProcedure table=testSplitTableRegion, region=c6c1c9ab03c5305a97a77e2634a5225c, ASSIGN; state=SPLITTING_NEW, location=jenkins-hbase6.apache.org,36983,1690200894572; forceNewPlan=false, retain=false 2023-07-24 12:16:44,937 INFO [jenkins-hbase6:44983] balancer.BaseLoadBalancer(1545): Reassigned 2 regions. 2 retained the pre-restart assignment. 2023-07-24 12:16:44,938 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=c6c1c9ab03c5305a97a77e2634a5225c, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:44,938 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690201004938"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690201004938"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690201004938"}]},"ts":"1690201004938"} 2023-07-24 12:16:44,940 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=149, ppid=147, state=RUNNABLE; OpenRegionProcedure c6c1c9ab03c5305a97a77e2634a5225c, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:16:44,942 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=8adba019c41c42da557e82cb7e9bb78f, regionState=OPENING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:44,942 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690201004942"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690201004942"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690201004942"}]},"ts":"1690201004942"} 2023-07-24 12:16:44,943 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=150, ppid=148, state=RUNNABLE; OpenRegionProcedure 8adba019c41c42da557e82cb7e9bb78f, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:16:45,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f. 2023-07-24 12:16:45,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => 8adba019c41c42da557e82cb7e9bb78f, NAME => 'testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.', STARTKEY => '41', ENDKEY => ''} 2023-07-24 12:16:45,094 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:16:45,094 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion 8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:45,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:16:45,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for 8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:45,095 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for 8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:45,096 INFO [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region 8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:45,097 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1 2023-07-24 12:16:45,097 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1 2023-07-24 12:16:45,097 INFO [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8adba019c41c42da557e82cb7e9bb78f columnFamilyName cf1 2023-07-24 12:16:45,109 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/0783f9a185a04aa795beb02ada67f44e.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/0783f9a185a04aa795beb02ada67f44e-top 2023-07-24 12:16:45,112 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/24600f1518f34667bfa83861f16713e9.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/24600f1518f34667bfa83861f16713e9-top 2023-07-24 12:16:45,115 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/45f2aabb72ed46698703826e2988bdb4.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/45f2aabb72ed46698703826e2988bdb4-top 2023-07-24 12:16:45,118 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/65b5fe2ba4194b79b8986167053580bc.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/65b5fe2ba4194b79b8986167053580bc-top 2023-07-24 12:16:45,121 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/6c423cf05c0e4689b54c93f830269180.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/6c423cf05c0e4689b54c93f830269180-top 2023-07-24 12:16:45,125 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/91feaa72255640fca41373b22989b49d.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/91feaa72255640fca41373b22989b49d-top 2023-07-24 12:16:45,128 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/d2111213b7c54ad581aa89ab4418b302.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/d2111213b7c54ad581aa89ab4418b302-top 2023-07-24 12:16:45,128 INFO [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(310): Store=8adba019c41c42da557e82cb7e9bb78f/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:45,128 INFO [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region 8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:45,129 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2 2023-07-24 12:16:45,129 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2 2023-07-24 12:16:45,129 INFO [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region 8adba019c41c42da557e82cb7e9bb78f columnFamilyName cf2 2023-07-24 12:16:45,141 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/24660271f70d4dd2a323b9099dacc4f7.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/24660271f70d4dd2a323b9099dacc4f7-top 2023-07-24 12:16:45,144 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/2714ecf62de54291ac8e9265bffe6eb0.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/2714ecf62de54291ac8e9265bffe6eb0-top 2023-07-24 12:16:45,148 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/330cd48cce594c7b98ec0f5724297c5e.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/330cd48cce594c7b98ec0f5724297c5e-top 2023-07-24 12:16:45,151 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/3552735b9780482baaaf6e26ae679da8.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/3552735b9780482baaaf6e26ae679da8-top 2023-07-24 12:16:45,154 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/a51cdb73005e402a8c2b8286e7a213c7.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/a51cdb73005e402a8c2b8286e7a213c7-top 2023-07-24 12:16:45,157 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/c7ab253de060407daadfee44e0e34624.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/c7ab253de060407daadfee44e0e34624-top 2023-07-24 12:16:45,160 DEBUG [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/e5aba3df13884f5399214e0bfb45d1c2.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/e5aba3df13884f5399214e0bfb45d1c2-top 2023-07-24 12:16:45,160 INFO [StoreOpener-8adba019c41c42da557e82cb7e9bb78f-1] regionserver.HStore(310): Store=8adba019c41c42da557e82cb7e9bb78f/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:45,161 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:45,162 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:45,163 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:16:45,164 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for 8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:45,165 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened 8adba019c41c42da557e82cb7e9bb78f; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=66030856, jitterRate=-0.016063570976257324}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:16:45,165 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for 8adba019c41c42da557e82cb7e9bb78f: 2023-07-24 12:16:45,166 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f., pid=150, masterSystemTime=1690201005091 2023-07-24 12:16:45,166 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:45,167 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 12:16:45,167 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:16:45,167 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:16:45,167 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f. 2023-07-24 12:16:45,167 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1912): 8adba019c41c42da557e82cb7e9bb78f/cf1 is initiating minor compaction (all files) 2023-07-24 12:16:45,167 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f. 2023-07-24 12:16:45,167 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of 8adba019c41c42da557e82cb7e9bb78f/cf1 in testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f. 2023-07-24 12:16:45,167 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1912): 8adba019c41c42da557e82cb7e9bb78f/cf2 is initiating minor compaction (all files) 2023-07-24 12:16:45,167 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegion(2259): Starting compaction of 8adba019c41c42da557e82cb7e9bb78f/cf2 in testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f. 2023-07-24 12:16:45,167 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/91feaa72255640fca41373b22989b49d.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/91feaa72255640fca41373b22989b49d-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/6c423cf05c0e4689b54c93f830269180.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/6c423cf05c0e4689b54c93f830269180-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/65b5fe2ba4194b79b8986167053580bc.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/65b5fe2ba4194b79b8986167053580bc-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/d2111213b7c54ad581aa89ab4418b302.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/d2111213b7c54ad581aa89ab4418b302-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/24600f1518f34667bfa83861f16713e9.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/24600f1518f34667bfa83861f16713e9-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/45f2aabb72ed46698703826e2988bdb4.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/45f2aabb72ed46698703826e2988bdb4-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/0783f9a185a04aa795beb02ada67f44e.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/0783f9a185a04aa795beb02ada67f44e-top] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/.tmp, totalSize=34.9 K 2023-07-24 12:16:45,168 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/3552735b9780482baaaf6e26ae679da8.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/3552735b9780482baaaf6e26ae679da8-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/e5aba3df13884f5399214e0bfb45d1c2.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/e5aba3df13884f5399214e0bfb45d1c2-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/c7ab253de060407daadfee44e0e34624.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/c7ab253de060407daadfee44e0e34624-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/2714ecf62de54291ac8e9265bffe6eb0.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/2714ecf62de54291ac8e9265bffe6eb0-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/24660271f70d4dd2a323b9099dacc4f7.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/24660271f70d4dd2a323b9099dacc4f7-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/330cd48cce594c7b98ec0f5724297c5e.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/330cd48cce594c7b98ec0f5724297c5e-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/a51cdb73005e402a8c2b8286e7a213c7.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/a51cdb73005e402a8c2b8286e7a213c7-top] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/.tmp, totalSize=34.9 K 2023-07-24 12:16:45,168 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 91feaa72255640fca41373b22989b49d.62e3147f93b8eed551ff46501d4053bf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690201002695 2023-07-24 12:16:45,168 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 3552735b9780482baaaf6e26ae679da8.62e3147f93b8eed551ff46501d4053bf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=7, earliestPutTs=1690201002695 2023-07-24 12:16:45,168 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 6c423cf05c0e4689b54c93f830269180.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690201002865 2023-07-24 12:16:45,168 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f. 2023-07-24 12:16:45,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f. 2023-07-24 12:16:45,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(130): Open testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c. 2023-07-24 12:16:45,169 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7854): Opening region: {ENCODED => c6c1c9ab03c5305a97a77e2634a5225c, NAME => 'testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.', STARTKEY => '', ENDKEY => '41'} 2023-07-24 12:16:45,169 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=148 updating hbase:meta row=8adba019c41c42da557e82cb7e9bb78f, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:45,169 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting e5aba3df13884f5399214e0bfb45d1c2.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=20, earliestPutTs=1690201002865 2023-07-24 12:16:45,169 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 65b5fe2ba4194b79b8986167053580bc.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690201003038 2023-07-24 12:16:45,169 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690201005169"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690201005169"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690201005169"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690201005169"}]},"ts":"1690201005169"} 2023-07-24 12:16:45,169 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(174): System coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro loaded, priority=536870911. 2023-07-24 12:16:45,169 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.MetricsRegionSourceImpl(79): Creating new MetricsRegionSourceImpl for table testSplitTableRegion c6c1c9ab03c5305a97a77e2634a5225c 2023-07-24 12:16:45,169 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(866): Instantiated testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.; StoreHotnessProtector, parallelPutToStoreThreadLimit=10 ; minColumnNum=100 ; preparePutThreadLimit=20 ; hotProtect now enable 2023-07-24 12:16:45,169 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7894): checking encryption for c6c1c9ab03c5305a97a77e2634a5225c 2023-07-24 12:16:45,169 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(7897): checking classloading for c6c1c9ab03c5305a97a77e2634a5225c 2023-07-24 12:16:45,170 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting c7ab253de060407daadfee44e0e34624.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=33, earliestPutTs=1690201003038 2023-07-24 12:16:45,170 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting d2111213b7c54ad581aa89ab4418b302.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690201003208 2023-07-24 12:16:45,170 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 2714ecf62de54291ac8e9265bffe6eb0.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=46, earliestPutTs=1690201003208 2023-07-24 12:16:45,171 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 24600f1518f34667bfa83861f16713e9.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1690201003382 2023-07-24 12:16:45,171 INFO [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf1 of region c6c1c9ab03c5305a97a77e2634a5225c 2023-07-24 12:16:45,171 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 24660271f70d4dd2a323b9099dacc4f7.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=59, earliestPutTs=1690201003382 2023-07-24 12:16:45,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=150, resume processing ppid=148 2023-07-24 12:16:45,171 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=150, ppid=148, state=SUCCESS; OpenRegionProcedure 8adba019c41c42da557e82cb7e9bb78f, server=jenkins-hbase6.apache.org,36983,1690200894572 in 227 msec 2023-07-24 12:16:45,171 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 45f2aabb72ed46698703826e2988bdb4.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1690201003555 2023-07-24 12:16:45,172 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 330cd48cce594c7b98ec0f5724297c5e.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=72, earliestPutTs=1690201003555 2023-07-24 12:16:45,172 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting a51cdb73005e402a8c2b8286e7a213c7.62e3147f93b8eed551ff46501d4053bf, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690201003731 2023-07-24 12:16:45,172 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 0783f9a185a04aa795beb02ada67f44e.62e3147f93b8eed551ff46501d4053bf, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=83, earliestPutTs=1690201003731 2023-07-24 12:16:45,172 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1 2023-07-24 12:16:45,172 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1 2023-07-24 12:16:45,173 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=148, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=8adba019c41c42da557e82cb7e9bb78f, ASSIGN in 386 msec 2023-07-24 12:16:45,173 INFO [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c6c1c9ab03c5305a97a77e2634a5225c columnFamilyName cf1 2023-07-24 12:16:45,183 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] throttle.PressureAwareThroughputController(145): 8adba019c41c42da557e82cb7e9bb78f#cf2#compaction#142 average throughput is 0.86 MB/second, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:16:45,183 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] throttle.PressureAwareThroughputController(145): 8adba019c41c42da557e82cb7e9bb78f#cf1#compaction#143 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:16:45,186 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/0783f9a185a04aa795beb02ada67f44e.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/0783f9a185a04aa795beb02ada67f44e-bottom 2023-07-24 12:16:45,189 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/24600f1518f34667bfa83861f16713e9.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/24600f1518f34667bfa83861f16713e9-bottom 2023-07-24 12:16:45,197 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/45f2aabb72ed46698703826e2988bdb4.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/45f2aabb72ed46698703826e2988bdb4-bottom 2023-07-24 12:16:45,202 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/.tmp/cf2/cf5e915b2a324e52a1566469b836e8c5 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/cf5e915b2a324e52a1566469b836e8c5 2023-07-24 12:16:45,204 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/.tmp/cf1/b67090c18c074c42a769f8428d5c7aa7 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/b67090c18c074c42a769f8428d5c7aa7 2023-07-24 12:16:45,206 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8adba019c41c42da557e82cb7e9bb78f/cf2 of 8adba019c41c42da557e82cb7e9bb78f into cf5e915b2a324e52a1566469b836e8c5(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:16:45,206 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8adba019c41c42da557e82cb7e9bb78f: 2023-07-24 12:16:45,206 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f., storeName=8adba019c41c42da557e82cb7e9bb78f/cf2, priority=9, startTime=1690201005167; duration=0sec 2023-07-24 12:16:45,206 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:45,206 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/65b5fe2ba4194b79b8986167053580bc.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/65b5fe2ba4194b79b8986167053580bc-bottom 2023-07-24 12:16:45,209 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in 8adba019c41c42da557e82cb7e9bb78f/cf1 of 8adba019c41c42da557e82cb7e9bb78f into b67090c18c074c42a769f8428d5c7aa7(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:16:45,209 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for 8adba019c41c42da557e82cb7e9bb78f: 2023-07-24 12:16:45,209 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f., storeName=8adba019c41c42da557e82cb7e9bb78f/cf1, priority=9, startTime=1690201005166; duration=0sec 2023-07-24 12:16:45,209 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:45,213 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/6c423cf05c0e4689b54c93f830269180.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/6c423cf05c0e4689b54c93f830269180-bottom 2023-07-24 12:16:45,215 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/91feaa72255640fca41373b22989b49d.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/91feaa72255640fca41373b22989b49d-bottom 2023-07-24 12:16:45,218 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/d2111213b7c54ad581aa89ab4418b302.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/d2111213b7c54ad581aa89ab4418b302-bottom 2023-07-24 12:16:45,218 INFO [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(310): Store=c6c1c9ab03c5305a97a77e2634a5225c/cf1, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:45,218 INFO [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(381): Created cacheConfig: cacheDataOnRead=true, cacheDataOnWrite=false, cacheIndexesOnWrite=false, cacheBloomsOnWrite=false, cacheEvictOnClose=false, cacheDataCompressed=false, prefetchOnOpen=false, for column family cf2 of region c6c1c9ab03c5305a97a77e2634a5225c 2023-07-24 12:16:45,219 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2 2023-07-24 12:16:45,219 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] util.CommonFSUtils(522): Set storagePolicy=HOT for path=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2 2023-07-24 12:16:45,219 INFO [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] compactions.CompactionConfiguration(173): size [minCompactSize:128 MB, maxCompactSize:8.00 EB, offPeakMaxCompactSize:8.00 EB); files [minFilesToCompact:3, maxFilesToCompact:10); ratio 1.200000; off-peak ratio 5.000000; throttle point 2684354560; major period 0, major jitter 0.500000, min locality to compact 0.000000; tiered compaction: max_age 9223372036854775807, incoming window min 6, compaction policy for tiered window org.apache.hadoop.hbase.regionserver.compactions.ExploringCompactionPolicy, single output for minor true, compaction window factory org.apache.hadoop.hbase.regionserver.compactions.ExponentialCompactionWindowFactory, region c6c1c9ab03c5305a97a77e2634a5225c columnFamilyName cf2 2023-07-24 12:16:45,231 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/24660271f70d4dd2a323b9099dacc4f7.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/24660271f70d4dd2a323b9099dacc4f7-bottom 2023-07-24 12:16:45,234 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/2714ecf62de54291ac8e9265bffe6eb0.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/2714ecf62de54291ac8e9265bffe6eb0-bottom 2023-07-24 12:16:45,237 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/330cd48cce594c7b98ec0f5724297c5e.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/330cd48cce594c7b98ec0f5724297c5e-bottom 2023-07-24 12:16:45,240 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/3552735b9780482baaaf6e26ae679da8.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/3552735b9780482baaaf6e26ae679da8-bottom 2023-07-24 12:16:45,242 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/a51cdb73005e402a8c2b8286e7a213c7.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/a51cdb73005e402a8c2b8286e7a213c7-bottom 2023-07-24 12:16:45,245 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/c7ab253de060407daadfee44e0e34624.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/c7ab253de060407daadfee44e0e34624-bottom 2023-07-24 12:16:45,248 DEBUG [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(539): loaded hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/e5aba3df13884f5399214e0bfb45d1c2.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/e5aba3df13884f5399214e0bfb45d1c2-bottom 2023-07-24 12:16:45,248 INFO [StoreOpener-c6c1c9ab03c5305a97a77e2634a5225c-1] regionserver.HStore(310): Store=c6c1c9ab03c5305a97a77e2634a5225c/cf2, memstore type=DefaultMemStore, storagePolicy=HOT, verifyBulkLoads=false, parallelPutCountPrintThreshold=50, encoding=NONE, compression=NONE 2023-07-24 12:16:45,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c 2023-07-24 12:16:45,249 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(5209): Found 0 recovered edits file(s) under hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c 2023-07-24 12:16:45,251 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.FlushLargeStoresPolicy(65): No hbase.hregion.percolumnfamilyflush.size.lower.bound set in table testSplitTableRegion descriptor;using region.getMemStoreFlushHeapSize/# of families (64.0 M)) instead. 2023-07-24 12:16:45,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1055): writing seq id for c6c1c9ab03c5305a97a77e2634a5225c 2023-07-24 12:16:45,252 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1072): Opened c6c1c9ab03c5305a97a77e2634a5225c; next sequenceid=86; SteppingSplitPolicysuper{IncreasingToUpperBoundRegionSplitPolicy{initialSize=268435456, ConstantSizeRegionSplitPolicy{desiredMaxFileSize=63111949, jitterRate=-0.059558674693107605}}}, FlushLargeStoresPolicy{flushSizeLowerBound=67108864} 2023-07-24 12:16:45,252 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(965): Region open journal for c6c1c9ab03c5305a97a77e2634a5225c: 2023-07-24 12:16:45,253 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2336): Post open deploy tasks for testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c., pid=149, masterSystemTime=1690201005091 2023-07-24 12:16:45,253 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:45,254 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.CompactSplit(385): Small Compaction requested: system; Because: Opening Region; compactionQueue=(longCompactions=0:shortCompactions=1), splitQueue=0 2023-07-24 12:16:45,254 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:16:45,254 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.SortedCompactionPolicy(75): Selecting compaction from 7 store files, 0 compacting, 7 eligible, 16 blocking 2023-07-24 12:16:45,254 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf2 since it belongs to recently split daughter region testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c. 2023-07-24 12:16:45,254 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1898): Keeping/Overriding Compaction request priority to -2147482648 for CF cf1 since it belongs to recently split daughter region testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c. 2023-07-24 12:16:45,254 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1912): c6c1c9ab03c5305a97a77e2634a5225c/cf2 is initiating minor compaction (all files) 2023-07-24 12:16:45,254 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1912): c6c1c9ab03c5305a97a77e2634a5225c/cf1 is initiating minor compaction (all files) 2023-07-24 12:16:45,254 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegion(2259): Starting compaction of c6c1c9ab03c5305a97a77e2634a5225c/cf2 in testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c. 2023-07-24 12:16:45,254 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegion(2259): Starting compaction of c6c1c9ab03c5305a97a77e2634a5225c/cf1 in testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c. 2023-07-24 12:16:45,255 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/3552735b9780482baaaf6e26ae679da8.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/3552735b9780482baaaf6e26ae679da8-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/e5aba3df13884f5399214e0bfb45d1c2.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/e5aba3df13884f5399214e0bfb45d1c2-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/c7ab253de060407daadfee44e0e34624.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/c7ab253de060407daadfee44e0e34624-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/2714ecf62de54291ac8e9265bffe6eb0.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/2714ecf62de54291ac8e9265bffe6eb0-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/24660271f70d4dd2a323b9099dacc4f7.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/24660271f70d4dd2a323b9099dacc4f7-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/330cd48cce594c7b98ec0f5724297c5e.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/330cd48cce594c7b98ec0f5724297c5e-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/a51cdb73005e402a8c2b8286e7a213c7.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/a51cdb73005e402a8c2b8286e7a213c7-bottom] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/.tmp, totalSize=34.9 K 2023-07-24 12:16:45,255 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1468): Starting compaction of [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/91feaa72255640fca41373b22989b49d.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/91feaa72255640fca41373b22989b49d-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/6c423cf05c0e4689b54c93f830269180.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/6c423cf05c0e4689b54c93f830269180-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/65b5fe2ba4194b79b8986167053580bc.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/65b5fe2ba4194b79b8986167053580bc-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/d2111213b7c54ad581aa89ab4418b302.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/d2111213b7c54ad581aa89ab4418b302-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/24600f1518f34667bfa83861f16713e9.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/24600f1518f34667bfa83861f16713e9-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/45f2aabb72ed46698703826e2988bdb4.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/45f2aabb72ed46698703826e2988bdb4-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/0783f9a185a04aa795beb02ada67f44e.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/0783f9a185a04aa795beb02ada67f44e-bottom] into tmpdir=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/.tmp, totalSize=34.9 K 2023-07-24 12:16:45,255 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 3552735b9780482baaaf6e26ae679da8.62e3147f93b8eed551ff46501d4053bf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690201002695 2023-07-24 12:16:45,255 DEBUG [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionServer(2363): Finished post open deploy task for testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c. 2023-07-24 12:16:45,255 INFO [RS_OPEN_REGION-regionserver/jenkins-hbase6:0-0] handler.AssignRegionHandler(158): Opened testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c. 2023-07-24 12:16:45,255 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 91feaa72255640fca41373b22989b49d.62e3147f93b8eed551ff46501d4053bf, keycount=1, bloomtype=ROW, size=4.8 K, encoding=NONE, compression=NONE, seqNum=6, earliestPutTs=1690201002695 2023-07-24 12:16:45,256 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=147 updating hbase:meta row=c6c1c9ab03c5305a97a77e2634a5225c, regionState=OPEN, openSeqNum=86, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:45,256 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":5,"row":"testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690201005256"},{"qualifier":"server","vlen":31,"tag":[],"timestamp":"1690201005256"},{"qualifier":"serverstartcode","vlen":8,"tag":[],"timestamp":"1690201005256"},{"qualifier":"seqnumDuringOpen","vlen":8,"tag":[],"timestamp":"1690201005256"}]},"ts":"1690201005256"} 2023-07-24 12:16:45,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=149, resume processing ppid=147 2023-07-24 12:16:45,258 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=149, ppid=147, state=SUCCESS; OpenRegionProcedure c6c1c9ab03c5305a97a77e2634a5225c, server=jenkins-hbase6.apache.org,36983,1690200894572 in 318 msec 2023-07-24 12:16:45,259 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting e5aba3df13884f5399214e0bfb45d1c2.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690201002865 2023-07-24 12:16:45,259 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=147, resume processing ppid=144 2023-07-24 12:16:45,259 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 6c423cf05c0e4689b54c93f830269180.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=19, earliestPutTs=1690201002865 2023-07-24 12:16:45,259 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=147, ppid=144, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=c6c1c9ab03c5305a97a77e2634a5225c, ASSIGN in 473 msec 2023-07-24 12:16:45,260 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting c7ab253de060407daadfee44e0e34624.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690201003038 2023-07-24 12:16:45,260 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 65b5fe2ba4194b79b8986167053580bc.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=32, earliestPutTs=1690201003038 2023-07-24 12:16:45,260 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 2714ecf62de54291ac8e9265bffe6eb0.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690201003208 2023-07-24 12:16:45,260 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting d2111213b7c54ad581aa89ab4418b302.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=45, earliestPutTs=1690201003208 2023-07-24 12:16:45,261 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 24660271f70d4dd2a323b9099dacc4f7.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690201003382 2023-07-24 12:16:45,261 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=144, state=SUCCESS; SplitTableRegionProcedure table=testSplitTableRegion, parent=62e3147f93b8eed551ff46501d4053bf, daughterA=c6c1c9ab03c5305a97a77e2634a5225c, daughterB=8adba019c41c42da557e82cb7e9bb78f in 1.5230 sec 2023-07-24 12:16:45,261 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 24600f1518f34667bfa83861f16713e9.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=58, earliestPutTs=1690201003382 2023-07-24 12:16:45,261 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting 330cd48cce594c7b98ec0f5724297c5e.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690201003555 2023-07-24 12:16:45,261 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 45f2aabb72ed46698703826e2988bdb4.62e3147f93b8eed551ff46501d4053bf, keycount=5, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=71, earliestPutTs=1690201003555 2023-07-24 12:16:45,261 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] compactions.Compactor(207): Compacting a51cdb73005e402a8c2b8286e7a213c7.62e3147f93b8eed551ff46501d4053bf, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690201003731 2023-07-24 12:16:45,261 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] compactions.Compactor(207): Compacting 0783f9a185a04aa795beb02ada67f44e.62e3147f93b8eed551ff46501d4053bf, keycount=4, bloomtype=ROW, size=5.0 K, encoding=NONE, compression=NONE, seqNum=82, earliestPutTs=1690201003731 2023-07-24 12:16:45,269 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] throttle.PressureAwareThroughputController(145): c6c1c9ab03c5305a97a77e2634a5225c#cf2#compaction#144 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 1 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:16:45,269 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] throttle.PressureAwareThroughputController(145): c6c1c9ab03c5305a97a77e2634a5225c#cf1#compaction#145 average throughput is unlimited, slept 0 time(s) and total slept time is 0 ms. 0 active operations remaining, total limit is 50.00 MB/second 2023-07-24 12:16:45,276 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=36983] ipc.CallRunner(144): callId: 1116 service: ClientService methodName: Scan size: 143 connection: 172.31.2.20:48798 deadline: 1690201065276, exception=org.apache.hadoop.hbase.NotServingRegionException: testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf. is not online on jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:45,284 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/.tmp/cf2/49d3a39f5c4f4d22a21c8f3a6d46e1cc as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/49d3a39f5c4f4d22a21c8f3a6d46e1cc 2023-07-24 12:16:45,284 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/.tmp/cf1/3c9ecb1dd2fc4e2f9dca8b060de80898 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/3c9ecb1dd2fc4e2f9dca8b060de80898 2023-07-24 12:16:45,288 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c6c1c9ab03c5305a97a77e2634a5225c/cf2 of c6c1c9ab03c5305a97a77e2634a5225c into 49d3a39f5c4f4d22a21c8f3a6d46e1cc(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:16:45,288 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HStore(1652): Completed compaction of 7 (all) file(s) in c6c1c9ab03c5305a97a77e2634a5225c/cf1 of c6c1c9ab03c5305a97a77e2634a5225c into 3c9ecb1dd2fc4e2f9dca8b060de80898(size=6.1 K), total size for store is 6.1 K. This selection was in queue for 0sec, and took 0sec to execute. 2023-07-24 12:16:45,288 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.HRegion(2289): Compaction status journal for c6c1c9ab03c5305a97a77e2634a5225c: 2023-07-24 12:16:45,288 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.HRegion(2289): Compaction status journal for c6c1c9ab03c5305a97a77e2634a5225c: 2023-07-24 12:16:45,288 INFO [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c., storeName=c6c1c9ab03c5305a97a77e2634a5225c/cf2, priority=9, startTime=1690201005254; duration=0sec 2023-07-24 12:16:45,288 INFO [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(627): Completed compaction region=testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c., storeName=c6c1c9ab03c5305a97a77e2634a5225c/cf1, priority=9, startTime=1690201005253; duration=0sec 2023-07-24 12:16:45,288 DEBUG [RS:2;jenkins-hbase6:36983-longCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:45,288 DEBUG [RS:2;jenkins-hbase6:36983-shortCompactions-0] regionserver.CompactSplit$CompactionRunner(659): Status compactionQueue=(longCompactions=0:shortCompactions=0), splitQueue=0 2023-07-24 12:16:45,386 DEBUG [Listener at localhost/37279] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-24 12:16:45,393 INFO [Listener at localhost/37279] assignment.TestSplitTableRegionProcedure(570): Row Count = 60 2023-07-24 12:16:45,394 DEBUG [Listener at localhost/37279] client.ClientScanner(240): Advancing internal scanner to startKey at '41', inclusive 2023-07-24 12:16:45,402 WARN [Listener at localhost/37279] procedure2.ProcedureTestingUtility(193): Set Kill before store update to: false 2023-07-24 12:16:45,403 INFO [Listener at localhost/37279] client.HBaseAdmin$15(890): Started disable of testSplitTableRegion 2023-07-24 12:16:45,403 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$11(2418): Client=jenkins//172.31.2.20 disable testSplitTableRegion 2023-07-24 12:16:45,404 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=151, state=RUNNABLE:DISABLE_TABLE_PREPARE; DisableTableProcedure table=testSplitTableRegion 2023-07-24 12:16:45,406 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-07-24 12:16:45,406 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690201005406"}]},"ts":"1690201005406"} 2023-07-24 12:16:45,407 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLING in hbase:meta 2023-07-24 12:16:45,409 INFO [PEWorker-1] procedure.DisableTableProcedure(293): Set testSplitTableRegion to state=DISABLING 2023-07-24 12:16:45,409 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=c6c1c9ab03c5305a97a77e2634a5225c, UNASSIGN}, {pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=8adba019c41c42da557e82cb7e9bb78f, UNASSIGN}] 2023-07-24 12:16:45,410 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=153, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=8adba019c41c42da557e82cb7e9bb78f, UNASSIGN 2023-07-24 12:16:45,410 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=8adba019c41c42da557e82cb7e9bb78f, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:45,411 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690201005410"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690201005410"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690201005410"}]},"ts":"1690201005410"} 2023-07-24 12:16:45,411 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=154, ppid=153, state=RUNNABLE; CloseRegionProcedure 8adba019c41c42da557e82cb7e9bb78f, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:16:45,413 INFO [PEWorker-1] procedure.MasterProcedureScheduler(727): Took xlock for pid=152, ppid=151, state=RUNNABLE:REGION_STATE_TRANSITION_CLOSE; TransitRegionStateProcedure table=testSplitTableRegion, region=c6c1c9ab03c5305a97a77e2634a5225c, UNASSIGN 2023-07-24 12:16:45,414 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=c6c1c9ab03c5305a97a77e2634a5225c, regionState=CLOSING, regionLocation=jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:45,414 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":3,"row":"testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690201005414"},{"qualifier":"sn","vlen":45,"tag":[],"timestamp":"1690201005414"},{"qualifier":"state","vlen":7,"tag":[],"timestamp":"1690201005414"}]},"ts":"1690201005414"} 2023-07-24 12:16:45,416 INFO [PEWorker-1] procedure2.ProcedureExecutor(1681): Initialized subprocedures=[{pid=155, ppid=152, state=RUNNABLE; CloseRegionProcedure c6c1c9ab03c5305a97a77e2634a5225c, server=jenkins-hbase6.apache.org,36983,1690200894572}] 2023-07-24 12:16:45,507 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-07-24 12:16:45,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close 8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:45,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 8adba019c41c42da557e82cb7e9bb78f, disabling compactions & flushes 2023-07-24 12:16:45,563 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f. 2023-07-24 12:16:45,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f. 2023-07-24 12:16:45,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f. after waiting 0 ms 2023-07-24 12:16:45,563 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f. 2023-07-24 12:16:45,564 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/91feaa72255640fca41373b22989b49d.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/91feaa72255640fca41373b22989b49d-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/6c423cf05c0e4689b54c93f830269180.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/6c423cf05c0e4689b54c93f830269180-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/65b5fe2ba4194b79b8986167053580bc.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/65b5fe2ba4194b79b8986167053580bc-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/d2111213b7c54ad581aa89ab4418b302.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/d2111213b7c54ad581aa89ab4418b302-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/24600f1518f34667bfa83861f16713e9.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/24600f1518f34667bfa83861f16713e9-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/45f2aabb72ed46698703826e2988bdb4.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/45f2aabb72ed46698703826e2988bdb4-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/0783f9a185a04aa795beb02ada67f44e.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/0783f9a185a04aa795beb02ada67f44e-top] to archive 2023-07-24 12:16:45,565 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:16:45,566 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/91feaa72255640fca41373b22989b49d.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/91feaa72255640fca41373b22989b49d.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,567 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/6c423cf05c0e4689b54c93f830269180.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/6c423cf05c0e4689b54c93f830269180.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,568 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/65b5fe2ba4194b79b8986167053580bc.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/65b5fe2ba4194b79b8986167053580bc.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,569 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/d2111213b7c54ad581aa89ab4418b302.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/d2111213b7c54ad581aa89ab4418b302.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,570 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/24600f1518f34667bfa83861f16713e9.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/24600f1518f34667bfa83861f16713e9.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,571 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/45f2aabb72ed46698703826e2988bdb4.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/45f2aabb72ed46698703826e2988bdb4.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,572 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/0783f9a185a04aa795beb02ada67f44e.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/0783f9a185a04aa795beb02ada67f44e.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,573 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/3552735b9780482baaaf6e26ae679da8.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/3552735b9780482baaaf6e26ae679da8-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/e5aba3df13884f5399214e0bfb45d1c2.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/e5aba3df13884f5399214e0bfb45d1c2-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/c7ab253de060407daadfee44e0e34624.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/c7ab253de060407daadfee44e0e34624-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/2714ecf62de54291ac8e9265bffe6eb0.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/2714ecf62de54291ac8e9265bffe6eb0-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/24660271f70d4dd2a323b9099dacc4f7.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/24660271f70d4dd2a323b9099dacc4f7-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/330cd48cce594c7b98ec0f5724297c5e.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/330cd48cce594c7b98ec0f5724297c5e-top, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/a51cdb73005e402a8c2b8286e7a213c7.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/a51cdb73005e402a8c2b8286e7a213c7-top] to archive 2023-07-24 12:16:45,574 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:16:45,575 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/3552735b9780482baaaf6e26ae679da8.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/3552735b9780482baaaf6e26ae679da8.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,576 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/e5aba3df13884f5399214e0bfb45d1c2.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/e5aba3df13884f5399214e0bfb45d1c2.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,577 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/c7ab253de060407daadfee44e0e34624.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/c7ab253de060407daadfee44e0e34624.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,578 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/2714ecf62de54291ac8e9265bffe6eb0.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/2714ecf62de54291ac8e9265bffe6eb0.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,579 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/24660271f70d4dd2a323b9099dacc4f7.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/24660271f70d4dd2a323b9099dacc4f7.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,580 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/330cd48cce594c7b98ec0f5724297c5e.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/330cd48cce594c7b98ec0f5724297c5e.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,581 DEBUG [StoreCloser-testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/a51cdb73005e402a8c2b8286e7a213c7.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/a51cdb73005e402a8c2b8286e7a213c7.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 12:16:45,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:16:45,584 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f. 2023-07-24 12:16:45,584 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 8adba019c41c42da557e82cb7e9bb78f: 2023-07-24 12:16:45,586 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed 8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:45,586 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(111): Close c6c1c9ab03c5305a97a77e2634a5225c 2023-07-24 12:16:45,586 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing c6c1c9ab03c5305a97a77e2634a5225c, disabling compactions & flushes 2023-07-24 12:16:45,586 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c. 2023-07-24 12:16:45,586 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c. 2023-07-24 12:16:45,586 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c. after waiting 0 ms 2023-07-24 12:16:45,586 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c. 2023-07-24 12:16:45,586 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=153 updating hbase:meta row=8adba019c41c42da557e82cb7e9bb78f, regionState=CLOSED 2023-07-24 12:16:45,586 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690201005586"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690201005586"}]},"ts":"1690201005586"} 2023-07-24 12:16:45,587 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/91feaa72255640fca41373b22989b49d.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/91feaa72255640fca41373b22989b49d-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/6c423cf05c0e4689b54c93f830269180.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/6c423cf05c0e4689b54c93f830269180-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/65b5fe2ba4194b79b8986167053580bc.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/65b5fe2ba4194b79b8986167053580bc-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/d2111213b7c54ad581aa89ab4418b302.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/d2111213b7c54ad581aa89ab4418b302-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/24600f1518f34667bfa83861f16713e9.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/24600f1518f34667bfa83861f16713e9-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/45f2aabb72ed46698703826e2988bdb4.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/45f2aabb72ed46698703826e2988bdb4-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/0783f9a185a04aa795beb02ada67f44e.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/0783f9a185a04aa795beb02ada67f44e-bottom] to archive 2023-07-24 12:16:45,587 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:16:45,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=154, resume processing ppid=153 2023-07-24 12:16:45,588 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=154, ppid=153, state=SUCCESS; CloseRegionProcedure 8adba019c41c42da557e82cb7e9bb78f, server=jenkins-hbase6.apache.org,36983,1690200894572 in 176 msec 2023-07-24 12:16:45,589 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/91feaa72255640fca41373b22989b49d.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/91feaa72255640fca41373b22989b49d.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,589 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=153, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=8adba019c41c42da557e82cb7e9bb78f, UNASSIGN in 179 msec 2023-07-24 12:16:45,590 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/6c423cf05c0e4689b54c93f830269180.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/6c423cf05c0e4689b54c93f830269180.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,591 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/65b5fe2ba4194b79b8986167053580bc.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/65b5fe2ba4194b79b8986167053580bc.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,592 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/d2111213b7c54ad581aa89ab4418b302.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/d2111213b7c54ad581aa89ab4418b302.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,593 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/24600f1518f34667bfa83861f16713e9.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/24600f1518f34667bfa83861f16713e9.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,594 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/45f2aabb72ed46698703826e2988bdb4.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/45f2aabb72ed46698703826e2988bdb4.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,595 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/0783f9a185a04aa795beb02ada67f44e.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/0783f9a185a04aa795beb02ada67f44e.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,597 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] regionserver.HStore(2712): Moving the files [hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/3552735b9780482baaaf6e26ae679da8.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/3552735b9780482baaaf6e26ae679da8-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/e5aba3df13884f5399214e0bfb45d1c2.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/e5aba3df13884f5399214e0bfb45d1c2-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/c7ab253de060407daadfee44e0e34624.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/c7ab253de060407daadfee44e0e34624-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/2714ecf62de54291ac8e9265bffe6eb0.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/2714ecf62de54291ac8e9265bffe6eb0-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/24660271f70d4dd2a323b9099dacc4f7.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/24660271f70d4dd2a323b9099dacc4f7-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/330cd48cce594c7b98ec0f5724297c5e.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/330cd48cce594c7b98ec0f5724297c5e-bottom, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/a51cdb73005e402a8c2b8286e7a213c7.62e3147f93b8eed551ff46501d4053bf->hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/a51cdb73005e402a8c2b8286e7a213c7-bottom] to archive 2023-07-24 12:16:45,597 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(360): Archiving compacted files. 2023-07-24 12:16:45,599 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/3552735b9780482baaaf6e26ae679da8.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/3552735b9780482baaaf6e26ae679da8.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,600 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/e5aba3df13884f5399214e0bfb45d1c2.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/e5aba3df13884f5399214e0bfb45d1c2.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,601 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/c7ab253de060407daadfee44e0e34624.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/c7ab253de060407daadfee44e0e34624.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,602 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/2714ecf62de54291ac8e9265bffe6eb0.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/2714ecf62de54291ac8e9265bffe6eb0.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,603 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/24660271f70d4dd2a323b9099dacc4f7.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/24660271f70d4dd2a323b9099dacc4f7.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,604 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/330cd48cce594c7b98ec0f5724297c5e.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/330cd48cce594c7b98ec0f5724297c5e.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,605 DEBUG [StoreCloser-testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.-1] backup.HFileArchiver(582): Archived from FileableStoreFile, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/a51cdb73005e402a8c2b8286e7a213c7.62e3147f93b8eed551ff46501d4053bf to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/a51cdb73005e402a8c2b8286e7a213c7.62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,607 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/recovered.edits/90.seqid, newMaxSeqId=90, maxSeqId=85 2023-07-24 12:16:45,608 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:16:45,608 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c. 2023-07-24 12:16:45,608 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for c6c1c9ab03c5305a97a77e2634a5225c: 2023-07-24 12:16:45,609 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.UnassignRegionHandler(149): Closed c6c1c9ab03c5305a97a77e2634a5225c 2023-07-24 12:16:45,609 INFO [PEWorker-1] assignment.RegionStateStore(219): pid=152 updating hbase:meta row=c6c1c9ab03c5305a97a77e2634a5225c, regionState=CLOSED 2023-07-24 12:16:45,609 DEBUG [PEWorker-1] assignment.RegionStateStore(405): Put {"totalColumns":2,"row":"testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.","families":{"info":[{"qualifier":"regioninfo","vlen":56,"tag":[],"timestamp":"1690201005609"},{"qualifier":"state","vlen":6,"tag":[],"timestamp":"1690201005609"}]},"ts":"1690201005609"} 2023-07-24 12:16:45,611 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=155, resume processing ppid=152 2023-07-24 12:16:45,612 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=155, ppid=152, state=SUCCESS; CloseRegionProcedure c6c1c9ab03c5305a97a77e2634a5225c, server=jenkins-hbase6.apache.org,36983,1690200894572 in 196 msec 2023-07-24 12:16:45,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1824): Finished subprocedure pid=152, resume processing ppid=151 2023-07-24 12:16:45,613 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=152, ppid=151, state=SUCCESS; TransitRegionStateProcedure table=testSplitTableRegion, region=c6c1c9ab03c5305a97a77e2634a5225c, UNASSIGN in 203 msec 2023-07-24 12:16:45,614 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Put {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":2,"tag":[],"timestamp":"1690201005614"}]},"ts":"1690201005614"} 2023-07-24 12:16:45,614 INFO [PEWorker-1] hbase.MetaTableAccessor(1635): Updated tableName=testSplitTableRegion, state=DISABLED in hbase:meta 2023-07-24 12:16:45,617 INFO [PEWorker-1] procedure.DisableTableProcedure(305): Set testSplitTableRegion to state=DISABLED 2023-07-24 12:16:45,618 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=151, state=SUCCESS; DisableTableProcedure table=testSplitTableRegion in 213 msec 2023-07-24 12:16:45,708 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=151 2023-07-24 12:16:45,709 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DISABLE, Table Name: default:testSplitTableRegion, procId: 151 completed 2023-07-24 12:16:45,709 INFO [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.HMaster$5(2228): Client=jenkins//172.31.2.20 delete testSplitTableRegion 2023-07-24 12:16:45,710 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] procedure2.ProcedureExecutor(1029): Stored pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION; DeleteTableProcedure table=testSplitTableRegion 2023-07-24 12:16:45,711 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(101): Waiting for RIT for pid=156, state=RUNNABLE:DELETE_TABLE_PRE_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-24 12:16:45,712 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-07-24 12:16:45,712 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(113): Deleting regions from filesystem for pid=156, state=RUNNABLE:DELETE_TABLE_CLEAR_FS_LAYOUT, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-24 12:16:45,715 DEBUG [HFileArchiver-13] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,715 DEBUG [HFileArchiver-15] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:45,715 DEBUG [HFileArchiver-14] backup.HFileArchiver(131): ARCHIVING hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c 2023-07-24 12:16:45,717 DEBUG [HFileArchiver-15] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/recovered.edits] 2023-07-24 12:16:45,717 DEBUG [HFileArchiver-13] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/recovered.edits] 2023-07-24 12:16:45,717 DEBUG [HFileArchiver-14] backup.HFileArchiver(159): Archiving [FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2, FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/recovered.edits] 2023-07-24 12:16:45,720 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/b67090c18c074c42a769f8428d5c7aa7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf1/b67090c18c074c42a769f8428d5c7aa7 2023-07-24 12:16:45,720 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/3c9ecb1dd2fc4e2f9dca8b060de80898 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf1/3c9ecb1dd2fc4e2f9dca8b060de80898 2023-07-24 12:16:45,721 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/0783f9a185a04aa795beb02ada67f44e to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/0783f9a185a04aa795beb02ada67f44e 2023-07-24 12:16:45,722 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/cf5e915b2a324e52a1566469b836e8c5 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/cf2/cf5e915b2a324e52a1566469b836e8c5 2023-07-24 12:16:45,723 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/49d3a39f5c4f4d22a21c8f3a6d46e1cc to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/cf2/49d3a39f5c4f4d22a21c8f3a6d46e1cc 2023-07-24 12:16:45,723 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/24600f1518f34667bfa83861f16713e9 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/24600f1518f34667bfa83861f16713e9 2023-07-24 12:16:45,724 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/45f2aabb72ed46698703826e2988bdb4 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/45f2aabb72ed46698703826e2988bdb4 2023-07-24 12:16:45,725 DEBUG [HFileArchiver-15] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/recovered.edits/90.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f/recovered.edits/90.seqid 2023-07-24 12:16:45,726 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/65b5fe2ba4194b79b8986167053580bc to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/65b5fe2ba4194b79b8986167053580bc 2023-07-24 12:16:45,726 DEBUG [HFileArchiver-14] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/recovered.edits/90.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c/recovered.edits/90.seqid 2023-07-24 12:16:45,726 DEBUG [HFileArchiver-15] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/8adba019c41c42da557e82cb7e9bb78f 2023-07-24 12:16:45,726 DEBUG [HFileArchiver-14] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/c6c1c9ab03c5305a97a77e2634a5225c 2023-07-24 12:16:45,727 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/6c423cf05c0e4689b54c93f830269180 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/6c423cf05c0e4689b54c93f830269180 2023-07-24 12:16:45,728 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/91feaa72255640fca41373b22989b49d to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/91feaa72255640fca41373b22989b49d 2023-07-24 12:16:45,729 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/d2111213b7c54ad581aa89ab4418b302 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf1/d2111213b7c54ad581aa89ab4418b302 2023-07-24 12:16:45,732 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/24660271f70d4dd2a323b9099dacc4f7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/24660271f70d4dd2a323b9099dacc4f7 2023-07-24 12:16:45,733 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/2714ecf62de54291ac8e9265bffe6eb0 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/2714ecf62de54291ac8e9265bffe6eb0 2023-07-24 12:16:45,734 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/330cd48cce594c7b98ec0f5724297c5e to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/330cd48cce594c7b98ec0f5724297c5e 2023-07-24 12:16:45,736 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/3552735b9780482baaaf6e26ae679da8 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/3552735b9780482baaaf6e26ae679da8 2023-07-24 12:16:45,737 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/a51cdb73005e402a8c2b8286e7a213c7 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/a51cdb73005e402a8c2b8286e7a213c7 2023-07-24 12:16:45,738 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/c7ab253de060407daadfee44e0e34624 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/c7ab253de060407daadfee44e0e34624 2023-07-24 12:16:45,739 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/e5aba3df13884f5399214e0bfb45d1c2 to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/cf2/e5aba3df13884f5399214e0bfb45d1c2 2023-07-24 12:16:45,741 DEBUG [HFileArchiver-13] backup.HFileArchiver(582): Archived from FileablePath, hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/recovered.edits/85.seqid to hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/archive/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf/recovered.edits/85.seqid 2023-07-24 12:16:45,742 DEBUG [HFileArchiver-13] backup.HFileArchiver(596): Deleted hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/.tmp/data/default/testSplitTableRegion/62e3147f93b8eed551ff46501d4053bf 2023-07-24 12:16:45,742 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(328): Archived testSplitTableRegion regions 2023-07-24 12:16:45,743 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(118): Deleting regions from META for pid=156, state=RUNNABLE:DELETE_TABLE_REMOVE_FROM_META, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-24 12:16:45,745 WARN [PEWorker-1] procedure.DeleteTableProcedure(384): Deleting some vestigial 3 rows of testSplitTableRegion from hbase:meta 2023-07-24 12:16:45,746 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(421): Removing 'testSplitTableRegion' descriptor. 2023-07-24 12:16:45,747 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(124): Deleting assignment state for pid=156, state=RUNNABLE:DELETE_TABLE_UNASSIGN_REGIONS, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-24 12:16:45,747 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(411): Removing 'testSplitTableRegion' from region states. 2023-07-24 12:16:45,747 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690201005747"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:45,747 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690201005747"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:45,748 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.","families":{"info":[{"qualifier":"","vlen":0,"tag":[],"timestamp":"1690201005747"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:45,749 INFO [PEWorker-1] hbase.MetaTableAccessor(1788): Deleted 3 regions from META 2023-07-24 12:16:45,749 DEBUG [PEWorker-1] hbase.MetaTableAccessor(1789): Deleted regions: [{ENCODED => 62e3147f93b8eed551ff46501d4053bf, NAME => 'testSplitTableRegion,,1690201002191.62e3147f93b8eed551ff46501d4053bf.', STARTKEY => '', ENDKEY => ''}, {ENCODED => c6c1c9ab03c5305a97a77e2634a5225c, NAME => 'testSplitTableRegion,,1690201003736.c6c1c9ab03c5305a97a77e2634a5225c.', STARTKEY => '', ENDKEY => '41'}, {ENCODED => 8adba019c41c42da557e82cb7e9bb78f, NAME => 'testSplitTableRegion,41,1690201003736.8adba019c41c42da557e82cb7e9bb78f.', STARTKEY => '41', ENDKEY => ''}] 2023-07-24 12:16:45,749 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(415): Marking 'testSplitTableRegion' as deleted. 2023-07-24 12:16:45,749 DEBUG [PEWorker-1] hbase.MetaTableAccessor(2093): Delete {"totalColumns":1,"row":"testSplitTableRegion","families":{"table":[{"qualifier":"state","vlen":0,"tag":[],"timestamp":"1690201005749"}]},"ts":"9223372036854775807"} 2023-07-24 12:16:45,750 INFO [PEWorker-1] hbase.MetaTableAccessor(1658): Deleted table testSplitTableRegion state from META 2023-07-24 12:16:45,752 DEBUG [PEWorker-1] procedure.DeleteTableProcedure(130): Finished pid=156, state=RUNNABLE:DELETE_TABLE_POST_OPERATION, locked=true; DeleteTableProcedure table=testSplitTableRegion 2023-07-24 12:16:45,753 INFO [PEWorker-1] procedure2.ProcedureExecutor(1410): Finished pid=156, state=SUCCESS; DeleteTableProcedure table=testSplitTableRegion in 43 msec 2023-07-24 12:16:45,813 DEBUG [RpcServer.default.FPBQ.Fifo.handler=2,queue=0,port=44983] master.MasterRpcServices(1230): Checking to see if procedure is done pid=156 2023-07-24 12:16:45,813 INFO [Listener at localhost/37279] client.HBaseAdmin$TableFuture(3541): Operation: DELETE, Table Name: default:testSplitTableRegion, procId: 156 completed 2023-07-24 12:16:45,827 INFO [Listener at localhost/37279] hbase.ResourceChecker(175): after: master.assignment.TestSplitTableRegionProcedure#testSplitTableRegion Thread=378 (was 375) - Thread LEAK? -, OpenFileDescriptor=832 (was 839), MaxFileDescriptor=60000 (was 60000), SystemLoadAverage=188 (was 169) - SystemLoadAverage LEAK? -, ProcessCount=168 (was 168), AvailableMemoryMB=1435 (was 1482) 2023-07-24 12:16:45,827 INFO [Listener at localhost/37279] hbase.HBaseTestingUtility(1286): Shutting down minicluster 2023-07-24 12:16:45,827 INFO [Listener at localhost/37279] client.ConnectionImplementation(1979): Closing master protocol: MasterService 2023-07-24 12:16:45,828 DEBUG [Listener at localhost/37279] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x50c18f6e to 127.0.0.1:58848 2023-07-24 12:16:45,828 DEBUG [Listener at localhost/37279] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 12:16:45,829 DEBUG [Listener at localhost/37279] util.JVMClusterUtil(237): Shutting down HBase Cluster 2023-07-24 12:16:45,829 DEBUG [Listener at localhost/37279] util.JVMClusterUtil(257): Found active master hash=212509468, stopped=false 2023-07-24 12:16:45,829 INFO [Listener at localhost/37279] master.ServerManager(901): Cluster shutdown requested of master=jenkins-hbase6.apache.org,44983,1690200893288 2023-07-24 12:16:45,830 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 12:16:45,830 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 12:16:45,831 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 12:16:45,831 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/running 2023-07-24 12:16:45,831 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 12:16:45,831 INFO [Listener at localhost/37279] procedure2.ProcedureExecutor(629): Stopping 2023-07-24 12:16:45,831 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 12:16:45,831 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 12:16:45,831 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 12:16:45,831 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/running 2023-07-24 12:16:45,831 DEBUG [Listener at localhost/37279] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x00a8cb5b to 127.0.0.1:58848 2023-07-24 12:16:45,831 DEBUG [Listener at localhost/37279] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 12:16:45,832 INFO [Listener at localhost/37279] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase6.apache.org,41789,1690200894488' ***** 2023-07-24 12:16:45,832 INFO [Listener at localhost/37279] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 12:16:45,832 INFO [Listener at localhost/37279] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase6.apache.org,34575,1690200894532' ***** 2023-07-24 12:16:45,832 INFO [Listener at localhost/37279] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 12:16:45,832 INFO [Listener at localhost/37279] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase6.apache.org,36983,1690200894572' ***** 2023-07-24 12:16:45,832 INFO [Listener at localhost/37279] regionserver.HRegionServer(2311): STOPPED: Shutdown requested 2023-07-24 12:16:45,832 INFO [RS:1;jenkins-hbase6:34575] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 12:16:45,832 INFO [RS:1;jenkins-hbase6:34575] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 12:16:45,832 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 12:16:45,832 INFO [RS:1;jenkins-hbase6:34575] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 12:16:45,833 INFO [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(3305): Received CLOSE for f3d7ba30038785f88ca0646c63b6c13d 2023-07-24 12:16:45,833 INFO [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(1144): stopping server jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:45,833 DEBUG [RS:1;jenkins-hbase6:34575] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x5c6607c3 to 127.0.0.1:58848 2023-07-24 12:16:45,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing f3d7ba30038785f88ca0646c63b6c13d, disabling compactions & flushes 2023-07-24 12:16:45,833 DEBUG [RS:1;jenkins-hbase6:34575] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 12:16:45,833 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d. 2023-07-24 12:16:45,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d. 2023-07-24 12:16:45,833 INFO [RS:0;jenkins-hbase6:41789] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 12:16:45,833 INFO [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-24 12:16:45,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d. after waiting 0 ms 2023-07-24 12:16:45,833 DEBUG [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(1478): Online Regions={f3d7ba30038785f88ca0646c63b6c13d=hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d.} 2023-07-24 12:16:45,833 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d. 2023-07-24 12:16:45,833 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2745): Flushing f3d7ba30038785f88ca0646c63b6c13d 1/1 column families, dataSize=78 B heapSize=488 B 2023-07-24 12:16:45,834 DEBUG [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(1504): Waiting on f3d7ba30038785f88ca0646c63b6c13d 2023-07-24 12:16:45,834 INFO [RS:2;jenkins-hbase6:36983] regionserver.HeapMemoryManager(220): Stopping 2023-07-24 12:16:45,834 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 12:16:45,834 INFO [RS:2;jenkins-hbase6:36983] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 12:16:45,834 INFO [RS:0;jenkins-hbase6:41789] flush.RegionServerFlushTableProcedureManager(117): Stopping region server flush procedure manager gracefully. 2023-07-24 12:16:45,834 INFO [RS:2;jenkins-hbase6:36983] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 12:16:45,834 INFO [RS:0;jenkins-hbase6:41789] snapshot.RegionServerSnapshotManager(137): Stopping RegionServerSnapshotManager gracefully. 2023-07-24 12:16:45,834 INFO [MemStoreFlusher.0] regionserver.MemStoreFlusher$FlushHandler(361): MemStoreFlusher.0 exiting 2023-07-24 12:16:45,835 INFO [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(1144): stopping server jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:45,834 INFO [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer(1144): stopping server jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:45,835 DEBUG [RS:2;jenkins-hbase6:36983] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x168a8e12 to 127.0.0.1:58848 2023-07-24 12:16:45,835 DEBUG [RS:0;jenkins-hbase6:41789] zookeeper.ReadOnlyZKClient(361): Close zookeeper connection 0x16efbeef to 127.0.0.1:58848 2023-07-24 12:16:45,835 DEBUG [RS:2;jenkins-hbase6:36983] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 12:16:45,835 DEBUG [RS:0;jenkins-hbase6:41789] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 12:16:45,835 INFO [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer(1170): stopping server jenkins-hbase6.apache.org,36983,1690200894572; all regions closed. 2023-07-24 12:16:45,835 INFO [RS:0;jenkins-hbase6:41789] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 12:16:45,835 INFO [RS:0;jenkins-hbase6:41789] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 12:16:45,835 INFO [RS:0;jenkins-hbase6:41789] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 12:16:45,835 INFO [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(3305): Received CLOSE for 1588230740 2023-07-24 12:16:45,835 INFO [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(1474): Waiting on 1 regions to close 2023-07-24 12:16:45,835 DEBUG [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(1478): Online Regions={1588230740=hbase:meta,,1.1588230740} 2023-07-24 12:16:45,835 DEBUG [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(1504): Waiting on 1588230740 2023-07-24 12:16:45,835 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1604): Closing 1588230740, disabling compactions & flushes 2023-07-24 12:16:45,836 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1626): Closing region hbase:meta,,1.1588230740 2023-07-24 12:16:45,836 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1647): Waiting without time limit for close lock on hbase:meta,,1.1588230740 2023-07-24 12:16:45,836 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1714): Acquired close lock on hbase:meta,,1.1588230740 after waiting 0 ms 2023-07-24 12:16:45,836 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1724): Updates disabled for region hbase:meta,,1.1588230740 2023-07-24 12:16:45,836 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2745): Flushing 1588230740 3/3 column families, dataSize=79.40 KB heapSize=125.77 KB 2023-07-24 12:16:45,847 DEBUG [RS:2;jenkins-hbase6:36983] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/oldWALs 2023-07-24 12:16:45,847 INFO [RS:2;jenkins-hbase6:36983] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase6.apache.org%2C36983%2C1690200894572:(num 1690200896087) 2023-07-24 12:16:45,848 DEBUG [RS:2;jenkins-hbase6:36983] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 12:16:45,848 INFO [RS:2;jenkins-hbase6:36983] regionserver.LeaseManager(133): Closed leases 2023-07-24 12:16:45,848 INFO [RS:2;jenkins-hbase6:36983] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase6:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-24 12:16:45,849 INFO [RS:2;jenkins-hbase6:36983] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 12:16:45,849 INFO [regionserver/jenkins-hbase6:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 12:16:45,849 INFO [RS:2;jenkins-hbase6:36983] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 12:16:45,849 INFO [RS:2;jenkins-hbase6:36983] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 12:16:45,850 INFO [RS:2;jenkins-hbase6:36983] ipc.NettyRpcServer(158): Stopping server on /172.31.2.20:36983 2023-07-24 12:16:45,856 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:45,856 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:45,856 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 12:16:45,856 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 12:16:45,856 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase6.apache.org,36983,1690200894572 2023-07-24 12:16:45,856 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 12:16:45,856 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 12:16:45,858 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=78 B at sequenceid=6 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/namespace/f3d7ba30038785f88ca0646c63b6c13d/.tmp/info/47923b3f6e2f4605952a12c3d026d088 2023-07-24 12:16:45,859 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase6.apache.org,36983,1690200894572] 2023-07-24 12:16:45,859 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase6.apache.org,36983,1690200894572; numProcessing=1 2023-07-24 12:16:45,861 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase6.apache.org,36983,1690200894572 already deleted, retry=false 2023-07-24 12:16:45,861 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase6.apache.org,36983,1690200894572 expired; onlineServers=2 2023-07-24 12:16:45,863 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/namespace/f3d7ba30038785f88ca0646c63b6c13d/.tmp/info/47923b3f6e2f4605952a12c3d026d088 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/namespace/f3d7ba30038785f88ca0646c63b6c13d/info/47923b3f6e2f4605952a12c3d026d088 2023-07-24 12:16:45,867 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/namespace/f3d7ba30038785f88ca0646c63b6c13d/info/47923b3f6e2f4605952a12c3d026d088, entries=2, sequenceid=6, filesize=4.8 K 2023-07-24 12:16:45,868 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~78 B/78, heapSize ~472 B/472, currentSize=0 B/0 for f3d7ba30038785f88ca0646c63b6c13d in 35ms, sequenceid=6, compaction requested=false 2023-07-24 12:16:45,874 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=71.29 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/.tmp/info/692e0dc3e13a47c9b1fe12f49fd835d6 2023-07-24 12:16:45,875 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/namespace/f3d7ba30038785f88ca0646c63b6c13d/recovered.edits/9.seqid, newMaxSeqId=9, maxSeqId=1 2023-07-24 12:16:45,876 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:16:45,877 INFO [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d. 2023-07-24 12:16:45,877 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for f3d7ba30038785f88ca0646c63b6c13d: 2023-07-24 12:16:45,877 DEBUG [RS_CLOSE_REGION-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed hbase:namespace,,1690200896472.f3d7ba30038785f88ca0646c63b6c13d. 2023-07-24 12:16:45,879 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 692e0dc3e13a47c9b1fe12f49fd835d6 2023-07-24 12:16:45,889 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=2.58 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/.tmp/rep_barrier/2fa290e0d5f64938a39f218ccd5ac534 2023-07-24 12:16:45,892 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2fa290e0d5f64938a39f218ccd5ac534 2023-07-24 12:16:45,907 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=5.53 KB at sequenceid=210 (bloomFilter=false), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/.tmp/table/c0a7c290b5414720993aae7ce18df718 2023-07-24 12:16:45,910 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c0a7c290b5414720993aae7ce18df718 2023-07-24 12:16:45,910 INFO [regionserver/jenkins-hbase6:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-24 12:16:45,910 INFO [regionserver/jenkins-hbase6:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-24 12:16:45,910 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/.tmp/info/692e0dc3e13a47c9b1fe12f49fd835d6 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/info/692e0dc3e13a47c9b1fe12f49fd835d6 2023-07-24 12:16:45,913 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 692e0dc3e13a47c9b1fe12f49fd835d6 2023-07-24 12:16:45,914 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/info/692e0dc3e13a47c9b1fe12f49fd835d6, entries=59, sequenceid=210, filesize=11.1 K 2023-07-24 12:16:45,914 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/.tmp/rep_barrier/2fa290e0d5f64938a39f218ccd5ac534 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/rep_barrier/2fa290e0d5f64938a39f218ccd5ac534 2023-07-24 12:16:45,926 INFO [regionserver/jenkins-hbase6:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 12:16:45,926 INFO [regionserver/jenkins-hbase6:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 12:16:45,926 INFO [regionserver/jenkins-hbase6:0.leaseChecker] regionserver.LeaseManager(133): Closed leases 2023-07-24 12:16:45,929 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 2fa290e0d5f64938a39f218ccd5ac534 2023-07-24 12:16:45,929 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/rep_barrier/2fa290e0d5f64938a39f218ccd5ac534, entries=24, sequenceid=210, filesize=7.5 K 2023-07-24 12:16:45,930 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/.tmp/table/c0a7c290b5414720993aae7ce18df718 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/table/c0a7c290b5414720993aae7ce18df718 2023-07-24 12:16:45,933 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for c0a7c290b5414720993aae7ce18df718 2023-07-24 12:16:45,933 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/table/c0a7c290b5414720993aae7ce18df718, entries=36, sequenceid=210, filesize=8.0 K 2023-07-24 12:16:45,934 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(2948): Finished flush of dataSize ~79.40 KB/81308, heapSize ~125.73 KB/128744, currentSize=0 B/0 for 1588230740 in 98ms, sequenceid=210, compaction requested=false 2023-07-24 12:16:45,939 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] wal.WALSplitUtil(408): Wrote file=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/data/hbase/meta/1588230740/recovered.edits/213.seqid, newMaxSeqId=213, maxSeqId=1 2023-07-24 12:16:45,940 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.master.assignment.TestSplitTableRegionProcedure$RegionServerHostingReplicaSlowOpenCopro 2023-07-24 12:16:45,940 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] coprocessor.CoprocessorHost(310): Stop coprocessor org.apache.hadoop.hbase.coprocessor.MultiRowMutationEndpoint 2023-07-24 12:16:45,941 INFO [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1838): Closed hbase:meta,,1.1588230740 2023-07-24 12:16:45,941 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] regionserver.HRegion(1558): Region close journal for 1588230740: 2023-07-24 12:16:45,941 DEBUG [RS_CLOSE_META-regionserver/jenkins-hbase6:0-0] handler.CloseRegionHandler(117): Closed hbase:meta,,1.1588230740 2023-07-24 12:16:45,959 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 12:16:45,959 INFO [RS:2;jenkins-hbase6:36983] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase6.apache.org,36983,1690200894572; zookeeper connection closed. 2023-07-24 12:16:45,959 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:36983-0x100becce0d60003, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 12:16:45,960 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@7802eba0] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@7802eba0 2023-07-24 12:16:45,990 INFO [regionserver/jenkins-hbase6:0.Chore.1] hbase.ScheduledChore(146): Chore: CompactionChecker was stopped 2023-07-24 12:16:45,990 INFO [regionserver/jenkins-hbase6:0.Chore.1] hbase.ScheduledChore(146): Chore: MemstoreFlusherChore was stopped 2023-07-24 12:16:46,034 INFO [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(1170): stopping server jenkins-hbase6.apache.org,34575,1690200894532; all regions closed. 2023-07-24 12:16:46,036 INFO [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(1170): stopping server jenkins-hbase6.apache.org,41789,1690200894488; all regions closed. 2023-07-24 12:16:46,043 DEBUG [RS:1;jenkins-hbase6:34575] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/oldWALs 2023-07-24 12:16:46,043 DEBUG [RS:0;jenkins-hbase6:41789] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/oldWALs 2023-07-24 12:16:46,043 INFO [RS:1;jenkins-hbase6:34575] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase6.apache.org%2C34575%2C1690200894532:(num 1690200896087) 2023-07-24 12:16:46,043 INFO [RS:0;jenkins-hbase6:41789] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase6.apache.org%2C41789%2C1690200894488.meta:.meta(num 1690200896206) 2023-07-24 12:16:46,043 DEBUG [RS:1;jenkins-hbase6:34575] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 12:16:46,043 INFO [RS:1;jenkins-hbase6:34575] regionserver.LeaseManager(133): Closed leases 2023-07-24 12:16:46,043 INFO [RS:1;jenkins-hbase6:34575] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase6:0 had [ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS, ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS] on shutdown 2023-07-24 12:16:46,044 INFO [RS:1;jenkins-hbase6:34575] regionserver.CompactSplit(434): Waiting for Split Thread to finish... 2023-07-24 12:16:46,044 INFO [RS:1;jenkins-hbase6:34575] regionserver.CompactSplit(434): Waiting for Large Compaction Thread to finish... 2023-07-24 12:16:46,044 INFO [regionserver/jenkins-hbase6:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 12:16:46,044 INFO [RS:1;jenkins-hbase6:34575] regionserver.CompactSplit(434): Waiting for Small Compaction Thread to finish... 2023-07-24 12:16:46,045 INFO [RS:1;jenkins-hbase6:34575] ipc.NettyRpcServer(158): Stopping server on /172.31.2.20:34575 2023-07-24 12:16:46,047 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:46,047 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 12:16:46,048 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase6.apache.org,34575,1690200894532 2023-07-24 12:16:46,048 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase6.apache.org,34575,1690200894532] 2023-07-24 12:16:46,048 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase6.apache.org,34575,1690200894532; numProcessing=2 2023-07-24 12:16:46,048 WARN [Close-WAL-Writer-0] asyncfs.FanOutOneBlockAsyncDFSOutputHelper(641): complete file /user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/WALs/jenkins-hbase6.apache.org,41789,1690200894488/jenkins-hbase6.apache.org%2C41789%2C1690200894488.1690200896087 not finished, retry = 0 2023-07-24 12:16:46,051 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase6.apache.org,34575,1690200894532 already deleted, retry=false 2023-07-24 12:16:46,051 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase6.apache.org,34575,1690200894532 expired; onlineServers=1 2023-07-24 12:16:46,149 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 12:16:46,149 INFO [RS:1;jenkins-hbase6:34575] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase6.apache.org,34575,1690200894532; zookeeper connection closed. 2023-07-24 12:16:46,149 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:34575-0x100becce0d60002, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 12:16:46,150 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@e843842] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@e843842 2023-07-24 12:16:46,151 DEBUG [RS:0;jenkins-hbase6:41789] wal.AbstractFSWAL(1028): Moved 1 WAL file(s) to /user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/oldWALs 2023-07-24 12:16:46,151 INFO [RS:0;jenkins-hbase6:41789] wal.AbstractFSWAL(1031): Closed WAL: AsyncFSWAL jenkins-hbase6.apache.org%2C41789%2C1690200894488:(num 1690200896087) 2023-07-24 12:16:46,151 DEBUG [RS:0;jenkins-hbase6:41789] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 12:16:46,151 INFO [RS:0;jenkins-hbase6:41789] regionserver.LeaseManager(133): Closed leases 2023-07-24 12:16:46,151 INFO [RS:0;jenkins-hbase6:41789] hbase.ChoreService(369): Chore service for: regionserver/jenkins-hbase6:0 had [ScheduledChore name=CompactedHFilesCleaner, period=120000, unit=MILLISECONDS, ScheduledChore name=CompactionThroughputTuner, period=60000, unit=MILLISECONDS] on shutdown 2023-07-24 12:16:46,151 INFO [regionserver/jenkins-hbase6:0.logRoller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 12:16:46,152 INFO [RS:0;jenkins-hbase6:41789] ipc.NettyRpcServer(158): Stopping server on /172.31.2.20:41789 2023-07-24 12:16:46,154 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/rs/jenkins-hbase6.apache.org,41789,1690200894488 2023-07-24 12:16:46,154 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase/rs 2023-07-24 12:16:46,154 INFO [RegionServerTracker-0] master.RegionServerTracker(179): RegionServer ephemeral node deleted, processing expiration [jenkins-hbase6.apache.org,41789,1690200894488] 2023-07-24 12:16:46,154 DEBUG [RegionServerTracker-0] master.DeadServer(103): Processing jenkins-hbase6.apache.org,41789,1690200894488; numProcessing=3 2023-07-24 12:16:46,156 DEBUG [RegionServerTracker-0] zookeeper.RecoverableZooKeeper(172): Node /hbase/draining/jenkins-hbase6.apache.org,41789,1690200894488 already deleted, retry=false 2023-07-24 12:16:46,156 INFO [RegionServerTracker-0] master.ServerManager(561): Cluster shutdown set; jenkins-hbase6.apache.org,41789,1690200894488 expired; onlineServers=0 2023-07-24 12:16:46,156 INFO [RegionServerTracker-0] regionserver.HRegionServer(2297): ***** STOPPING region server 'jenkins-hbase6.apache.org,44983,1690200893288' ***** 2023-07-24 12:16:46,156 INFO [RegionServerTracker-0] regionserver.HRegionServer(2311): STOPPED: Cluster shutdown set; onlineServer=0 2023-07-24 12:16:46,157 DEBUG [M:0;jenkins-hbase6:44983] ipc.AbstractRpcClient(190): Codec=org.apache.hadoop.hbase.codec.KeyValueCodec@4d97a927, compressor=null, tcpKeepAlive=true, tcpNoDelay=true, connectTO=10000, readTO=20000, writeTO=60000, minIdleTimeBeforeClose=120000, maxRetries=0, fallbackAllowed=true, bind address=jenkins-hbase6.apache.org/172.31.2.20:0 2023-07-24 12:16:46,157 INFO [M:0;jenkins-hbase6:44983] regionserver.HRegionServer(1144): stopping server jenkins-hbase6.apache.org,44983,1690200893288 2023-07-24 12:16:46,157 INFO [M:0;jenkins-hbase6:44983] regionserver.HRegionServer(1170): stopping server jenkins-hbase6.apache.org,44983,1690200893288; all regions closed. 2023-07-24 12:16:46,157 DEBUG [M:0;jenkins-hbase6:44983] ipc.AbstractRpcClient(494): Stopping rpc client 2023-07-24 12:16:46,157 DEBUG [M:0;jenkins-hbase6:44983] cleaner.LogCleaner(198): Cancelling LogCleaner 2023-07-24 12:16:46,157 WARN [OldWALsCleaner-0] cleaner.LogCleaner(186): Interrupted while cleaning old WALs, will try to clean it next round. Exiting. 2023-07-24 12:16:46,157 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-HFileCleaner.large.0-1690200895697] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase6:0:becomeActiveMaster-HFileCleaner.large.0-1690200895697,5,FailOnTimeoutGroup] 2023-07-24 12:16:46,157 DEBUG [master/jenkins-hbase6:0:becomeActiveMaster-HFileCleaner.small.0-1690200895699] cleaner.HFileCleaner(288): Exit Thread[master/jenkins-hbase6:0:becomeActiveMaster-HFileCleaner.small.0-1690200895699,5,FailOnTimeoutGroup] 2023-07-24 12:16:46,157 DEBUG [M:0;jenkins-hbase6:44983] cleaner.HFileCleaner(317): Stopping file delete threads 2023-07-24 12:16:46,157 INFO [M:0;jenkins-hbase6:44983] master.MasterMobCompactionThread(168): Waiting for Mob Compaction Thread to finish... 2023-07-24 12:16:46,157 INFO [M:0;jenkins-hbase6:44983] master.MasterMobCompactionThread(168): Waiting for Region Server Mob Compaction Thread to finish... 2023-07-24 12:16:46,158 INFO [M:0;jenkins-hbase6:44983] hbase.ChoreService(369): Chore service for: master/jenkins-hbase6:0 had [] on shutdown 2023-07-24 12:16:46,158 DEBUG [M:0;jenkins-hbase6:44983] master.HMaster(1512): Stopping service threads 2023-07-24 12:16:46,158 INFO [M:0;jenkins-hbase6:44983] procedure2.RemoteProcedureDispatcher(119): Stopping procedure remote dispatcher 2023-07-24 12:16:46,158 ERROR [M:0;jenkins-hbase6:44983] procedure2.ProcedureExecutor(653): ThreadGroup java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] contains running threads; null: See STDOUT java.lang.ThreadGroup[name=PEWorkerGroup,maxpri=10] Thread[HFileArchiver-13,5,PEWorkerGroup] Thread[HFileArchiver-14,5,PEWorkerGroup] Thread[HFileArchiver-15,5,PEWorkerGroup] Thread[HFileArchiver-16,5,PEWorkerGroup] 2023-07-24 12:16:46,158 INFO [M:0;jenkins-hbase6:44983] region.RegionProcedureStore(113): Stopping the Region Procedure Store, isAbort=false 2023-07-24 12:16:46,158 DEBUG [normalizer-worker-0] normalizer.RegionNormalizerWorker(174): interrupt detected. terminating. 2023-07-24 12:16:46,159 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeDeleted, state=SyncConnected, path=/hbase/master 2023-07-24 12:16:46,159 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=NodeChildrenChanged, state=SyncConnected, path=/hbase 2023-07-24 12:16:46,159 DEBUG [M:0;jenkins-hbase6:44983] zookeeper.ZKUtil(398): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Unable to get data of znode /hbase/master because node does not exist (not an error) 2023-07-24 12:16:46,160 WARN [M:0;jenkins-hbase6:44983] master.ActiveMasterManager(326): Failed get of master address: java.io.IOException: Can't get master address from ZooKeeper; znode data == null 2023-07-24 12:16:46,160 INFO [M:0;jenkins-hbase6:44983] assignment.AssignmentManager(315): Stopping assignment manager 2023-07-24 12:16:46,160 DEBUG [zk-event-processor-pool-0] zookeeper.ZKUtil(164): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Set watcher on znode that does not yet exist, /hbase/master 2023-07-24 12:16:46,160 INFO [M:0;jenkins-hbase6:44983] region.MasterRegion(167): Closing local region {ENCODED => 1595e783b53d99cd5eef43b6debb2682, NAME => 'master:store,,1.1595e783b53d99cd5eef43b6debb2682.', STARTKEY => '', ENDKEY => ''}, isAbort=false 2023-07-24 12:16:46,160 DEBUG [M:0;jenkins-hbase6:44983] regionserver.HRegion(1604): Closing 1595e783b53d99cd5eef43b6debb2682, disabling compactions & flushes 2023-07-24 12:16:46,160 INFO [M:0;jenkins-hbase6:44983] regionserver.HRegion(1626): Closing region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 12:16:46,160 DEBUG [M:0;jenkins-hbase6:44983] regionserver.HRegion(1647): Waiting without time limit for close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 12:16:46,160 DEBUG [M:0;jenkins-hbase6:44983] regionserver.HRegion(1714): Acquired close lock on master:store,,1.1595e783b53d99cd5eef43b6debb2682. after waiting 0 ms 2023-07-24 12:16:46,160 DEBUG [M:0;jenkins-hbase6:44983] regionserver.HRegion(1724): Updates disabled for region master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 12:16:46,160 INFO [M:0;jenkins-hbase6:44983] regionserver.HRegion(2745): Flushing 1595e783b53d99cd5eef43b6debb2682 1/1 column families, dataSize=607.95 KB heapSize=718.95 KB 2023-07-24 12:16:46,190 INFO [M:0;jenkins-hbase6:44983] regionserver.DefaultStoreFlusher(82): Flushed memstore data size=607.95 KB at sequenceid=1282 (bloomFilter=true), to=hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/1dcd7c519a5541a38e076f09ab560bf1 2023-07-24 12:16:46,193 INFO [M:0;jenkins-hbase6:44983] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1dcd7c519a5541a38e076f09ab560bf1 2023-07-24 12:16:46,193 DEBUG [M:0;jenkins-hbase6:44983] regionserver.HRegionFileSystem(485): Committing hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/.tmp/proc/1dcd7c519a5541a38e076f09ab560bf1 as hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/1dcd7c519a5541a38e076f09ab560bf1 2023-07-24 12:16:46,196 INFO [M:0;jenkins-hbase6:44983] regionserver.StoreFileReader(520): Loaded Delete Family Bloom (CompoundBloomFilter) metadata for 1dcd7c519a5541a38e076f09ab560bf1 2023-07-24 12:16:46,196 INFO [M:0;jenkins-hbase6:44983] regionserver.HStore(1080): Added hdfs://localhost:33821/user/jenkins/test-data/b7506743-ef4b-57c2-cde9-0f3d4fe327ef/MasterData/data/master/store/1595e783b53d99cd5eef43b6debb2682/proc/1dcd7c519a5541a38e076f09ab560bf1, entries=156, sequenceid=1282, filesize=34.9 K 2023-07-24 12:16:46,197 INFO [M:0;jenkins-hbase6:44983] regionserver.HRegion(2948): Finished flush of dataSize ~607.95 KB/622544, heapSize ~718.94 KB/736192, currentSize=0 B/0 for 1595e783b53d99cd5eef43b6debb2682 in 37ms, sequenceid=1282, compaction requested=false 2023-07-24 12:16:46,198 INFO [M:0;jenkins-hbase6:44983] regionserver.HRegion(1838): Closed master:store,,1.1595e783b53d99cd5eef43b6debb2682. 2023-07-24 12:16:46,198 DEBUG [M:0;jenkins-hbase6:44983] regionserver.HRegion(1558): Region close journal for 1595e783b53d99cd5eef43b6debb2682: 2023-07-24 12:16:46,203 INFO [M:0;jenkins-hbase6:44983] flush.MasterFlushTableProcedureManager(83): stop: server shutting down. 2023-07-24 12:16:46,203 INFO [master:store-WAL-Roller] wal.AbstractWALRoller(243): LogRoller exiting. 2023-07-24 12:16:46,203 INFO [M:0;jenkins-hbase6:44983] ipc.NettyRpcServer(158): Stopping server on /172.31.2.20:44983 2023-07-24 12:16:46,205 DEBUG [M:0;jenkins-hbase6:44983] zookeeper.RecoverableZooKeeper(172): Node /hbase/rs/jenkins-hbase6.apache.org,44983,1690200893288 already deleted, retry=false 2023-07-24 12:16:46,255 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 12:16:46,255 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): regionserver:41789-0x100becce0d60001, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 12:16:46,255 INFO [RS:0;jenkins-hbase6:41789] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase6.apache.org,41789,1690200894488; zookeeper connection closed. 2023-07-24 12:16:46,256 INFO [Shutdown of org.apache.hadoop.hbase.fs.HFileSystem@9bde17e] hbase.MiniHBaseCluster$SingleFileSystemShutdownThread(215): Hook closing fs=org.apache.hadoop.hbase.fs.HFileSystem@9bde17e 2023-07-24 12:16:46,256 INFO [Listener at localhost/37279] util.JVMClusterUtil(335): Shutdown of 1 master(s) and 3 regionserver(s) complete 2023-07-24 12:16:46,356 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 12:16:46,356 INFO [M:0;jenkins-hbase6:44983] regionserver.HRegionServer(1227): Exiting; stopping=jenkins-hbase6.apache.org,44983,1690200893288; zookeeper connection closed. 2023-07-24 12:16:46,356 DEBUG [Listener at localhost/37279-EventThread] zookeeper.ZKWatcher(600): master:44983-0x100becce0d60000, quorum=127.0.0.1:58848, baseZNode=/hbase Received ZooKeeper Event, type=None, state=Closed, path=null 2023-07-24 12:16:46,358 WARN [Listener at localhost/37279] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-24 12:16:46,362 INFO [Listener at localhost/37279] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-24 12:16:46,464 WARN [BP-51740533-172.31.2.20-1690200889613 heartbeating to localhost/127.0.0.1:33821] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-51740533-172.31.2.20-1690200889613 (Datanode Uuid 8ac734ec-00fd-4136-ae08-8d364bd1fe4b) service to localhost/127.0.0.1:33821 2023-07-24 12:16:46,466 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/dfs/data/data5/current/BP-51740533-172.31.2.20-1690200889613] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 12:16:46,467 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/dfs/data/data6/current/BP-51740533-172.31.2.20-1690200889613] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 12:16:46,469 WARN [Listener at localhost/37279] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-24 12:16:46,471 INFO [Listener at localhost/37279] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-24 12:16:46,576 WARN [BP-51740533-172.31.2.20-1690200889613 heartbeating to localhost/127.0.0.1:33821] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-24 12:16:46,576 WARN [BP-51740533-172.31.2.20-1690200889613 heartbeating to localhost/127.0.0.1:33821] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-51740533-172.31.2.20-1690200889613 (Datanode Uuid dce5ccf6-5af1-4cab-b72f-523ad9084719) service to localhost/127.0.0.1:33821 2023-07-24 12:16:46,577 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/dfs/data/data3/current/BP-51740533-172.31.2.20-1690200889613] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 12:16:46,577 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/dfs/data/data4/current/BP-51740533-172.31.2.20-1690200889613] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 12:16:46,580 WARN [Listener at localhost/37279] datanode.DirectoryScanner(534): DirectoryScanner: shutdown has been called 2023-07-24 12:16:46,582 INFO [Listener at localhost/37279] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-24 12:16:46,689 WARN [BP-51740533-172.31.2.20-1690200889613 heartbeating to localhost/127.0.0.1:33821] datanode.IncrementalBlockReportManager(160): IncrementalBlockReportManager interrupted 2023-07-24 12:16:46,689 WARN [BP-51740533-172.31.2.20-1690200889613 heartbeating to localhost/127.0.0.1:33821] datanode.BPServiceActor(857): Ending block pool service for: Block pool BP-51740533-172.31.2.20-1690200889613 (Datanode Uuid 295fb89a-ce04-4534-9158-8d1eb7f1a7e1) service to localhost/127.0.0.1:33821 2023-07-24 12:16:46,690 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/dfs/data/data1/current/BP-51740533-172.31.2.20-1690200889613] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 12:16:46,691 WARN [refreshUsed-/home/jenkins/jenkins-home/workspace/HBase-Flaky-Tests_branch-2.4/hbase-server/target/test-data/8550272a-da13-0c67-9ea1-2860adfac234/cluster_da4e35df-dd99-af08-9c99-dd3fd8509b5d/dfs/data/data2/current/BP-51740533-172.31.2.20-1690200889613] fs.CachingGetSpaceUsed$RefreshThread(183): Thread Interrupted waiting to refresh disk information: sleep interrupted 2023-07-24 12:16:46,712 INFO [Listener at localhost/37279] log.Slf4jLog(67): Stopped HttpServer2$SelectChannelConnectorWithSafeStartup@localhost:0 2023-07-24 12:16:46,904 INFO [Listener at localhost/37279] zookeeper.MiniZooKeeperCluster(344): Shutdown MiniZK cluster with all ZK servers 2023-07-24 12:16:47,143 INFO [Listener at localhost/37279] hbase.HBaseTestingUtility(1293): Minicluster is down